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Executive Summary 

Oregon postsecondary institutions offer a variety of opportunities to students seeking programs in 
engineering. Most of these institutions offer pre-engineering coursework to students that will eventually 
transfer to other Oregon institutions and complete a degree in engineering. However, what is expected of 
students in the first two years of engineering studies is not explicit and varies by institution. This is due in 
part to the fact that the community colleges and OUS institutions have not identified a common 
engineering curriculum in terms of educational learning outcomes expected during those first years of 
study. Without explicit standards, it is difficult for high schools and postsecondary institutions to 
communicate expectations to students. 

This study undertook a standards development process designed to yield the first set of explicit pre- 
engineering learning outcomes that could then used to align high school and postsecondary, entry-level 
engineering courses. The process led to the development of learning outcomes for content knowledge, key 
cognitive strategies, and teaching practices that reflect the content and intellectual goals of best practice in 
pre-engineering education. 

Once these learning outcomes were developed, instructors provided course reviews to determine 
alignment to the draft outcomes and to identify the expectations of pre-engineering students in their entry- 
level courses. Instructors generally found that students in Oregon postsecondary pre-engineering 
programs enter courses with adequate knowledge and are expected to begin college study in engineering 
with a basic understanding of course content. 

In addition, instructors provide feedback on the importance of the draft learning outcomes for success in 
their courses as well as for preparation for upper-division engineering studies. Two-thirds of learning 
outcomes were identified as more or most important for the success in the first two years of engineering 
programs. Almost 40% of the learning outcomes are more or most important for successful entry into 
upper-division engineering courses. 

EPIC is pleased to provide this report which includes a list of the Oregon Pre-engineering Learning 
Outcomes as developed by reviewers and Oregon postsecondary educators. Also included are instructors’ 
reviews of the pre-engineering courses they teach along with feedback from follow-up telephone 
interviews with engineering instructors. Finally, a website has been developed to provide information 
regarding this study and resources for careers and education in engineering. 
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Educational Policy Improvement Center (EPIC) 

The Educational Policy Improvement Center, a 501(c)3 nonprofit organization, seeks to help policy 
makers and policy implementers alike do a better job of using educational policy as a tool to improve 
schooling and student learning. 

EPIC works with federal agencies, state education departments, non-governmental organizations, private 
foundations, and school districts to support research on a range of issues in the areas of high school-to- 
college articulation, adequacy funding, large-scale assessment models, and other policy initiatives 
designed to improve student success. 

On this study, EPIC worked in partnership with the Center for Educational Policy Research (CEPR), a 
University of Oregon research center with expertise in validity studies and high school-college 
articulation issues. 
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Overview 


Oregon Pre-engineering Learning Outcomes Project 
and EPiC 

The Oregon Pre-engineering Learning Outcomes Project was conducted by the Educational Policy 
Improvement Center (EPIC) with grant funding from the Engineering and Technology Industry Council 
(ETIC). The study sought to improve student preparation and success in pre-engineering programs 
through the development of the Oregon Pre-engineering Learning Outcomes, a comprehensive set of 
standards reflecting student learning, key cognitive strategies, and teaching practices essential to student 
success in the first two years of postsecondary studies in engineering programs at Oregon community 
colleges and Oregon University System (OUS) institutions. The learning outcomes are the first research- 
based standards of their kind developed for the state of Oregon. This study applied the learning outcomes 
to virtually all 100- and 200- level core-engineering courses to learn what is taught in courses statewide 
and to highlight alignment and gaps among institutions’ programs. These results are intended to assist the 
OUS in complying with recent legislation that mandates creation of a system of seamless credit transfer 
based on a common, statewide framework of learning expectations. 


Study Objectives and Applications 

In 2005, ETIC convened the Oregon Pre-engineering & Applied Sciences (OP AS) Strategic Planning 
Summit. This meeting of academic and industry minds identified five critical action areas in Oregon 
science, technology, engineering, and mathematics (STEM) education: motivation, preparation, transfer, 
retention, and diversity. This study addresses three of the OP AS committee goals as listed in the OP AS 
Summit Report (Table 1). 


Table 1; OPAS Summit Goals and Study Objectives 


Summit Goal 

OPAS Study Objective 

Use a common set of achievement and preparation 
standards. 

Establish coordinated and clearly articulated 
educational pathways, allowing students to plan and 
transition among Oregon educational institutions. 

Increase the number of Oregonians receiving two- 
year and four-year degrees in engineering and 
applied science. 

Develop Oregon Pre-engineering Learning 
Outcomes based on best practice research from 
national and state organizations 

Understand the knowledge and skills currently 
being taught in 100- and 200- level core courses in 
engineering programs. 

Develop a website that can provide information to 
students and educators on engineering education 
and careers. 
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Potential applications of the study 

Identifying learning expectations for the first two years of postsecondary engineering studies improves 
the preparation of high school students for college studies, provides a reference point for transferring 
between postsecondary engineering programs, and provides a baseline for upper-division engineering 
programs. 

Improve high school student preparation for engineering by clarifying the knowledge and skills they 
need to succeed in engineering programs at Oregon postsecondary institutions. With an improved 
understanding of the content of 100- and 200-level pre-engineering courses, ETIC and OP AS steering 
committees could develop materials for Oregon high schools that will enable more high school students to 
enter college prepared to succeed in engineering courses and to provide such information directly to high 
schools. The findings may influence current efforts to review and revise Oregon K-12 standards and 
assessments. The timing of the study is well suited to result in a significant impact on high school courses 
throughout the state. Because this project is the first of its kind to develop research-based standards for 
the knowledge and skills necessary for the success of all students seeking to study engineering at OUS 
institutions, it will inform state efforts to move high schools further in the direction of standards-based 
teaching and learning. 

Provide a reference point to postsecondary institutions when reviewing the content and focus of their 
engineering studies curriculum. Providing a common set of learning outcomes for all Oregon 
engineering studies may assist the OUS to comply with recent legislation that mandates creation of a 
system of seamless credit transfer based on a common, statewide framework of learning expectations. 

Improve student preparation for upper-division engineering study. Improving transparency of expected 
learning in lower-division engineering studies may increase student preparation for upper-division 
courses and define explicitly the foundational skills required for more advanced study in engineering. 
Increased retention and graduation rates from Oregon engineering programs may be a result of these 
efforts. 
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Methodology 


Overview of Methodology 

The study began with a review of best practices in pre-engineering standards and applied a modified 
Delphi approach to develop learning outcomes for pre-engineering courses in Oregon. A criterion-based 
analysis methodology identified what is taught in 100- and 200-level courses required of students seeking 
engineering degrees. Instructors’ ratings of their own courses were supplemented by trained raters’ 
evaluation of course documents. Engineering instructors were interviewed to further clarify findings. 

Specific Research Questions 

1 . According to best practice research, what are key learning outcomes for content 
knowledge, cognitive skills, and teaching practice expected of students completing the 
first two years of engineering education? 

2 . Which learning outcomes are taught during the first two years of engineering programs 
in Oregon universities and community colleges? 

Learning Outcomes Development 

EPIC researchers developed learning outcomes from research on best practice in engineering education. 
Researchers reviewed over 100 websites of national organizations, state education departments, and other 
engineering-related educational organizations for standards containing engineering content knowledge, 
key cognitive strategies, and teaching practices. Appendix G provides a comprehensive list of these 
sources. This research culminated in a draft list of 14 topical areas and 88 performance expectations 
across the three groups of standards. 

Distinguished reviewers were recruited to review, edit, and endorse the standards, and EPIC conducted 
two sequential reviews. Reviewers were unpaid and volunteered about an hour per review. 

Recruitment of distinguished engineering educators 

Researchers reviewed websites of Oregon postsecondary institutions, national institutions listed as being 
in the top 20 engineering programs, and national engineering organizations in order to identify potential 
reviewers to edit and endorse the initial draft of learning outcomes. Individuals were considered 
distinguished if they met at least one of the following criteria: 
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♦ Prestige/status or acknowledged expertise in a field of engineering 

♦ Author of publication/report related to improving undergraduate teaching 

♦ Member of an engineering-related national group/committee/organization 

♦ Recipient of engineering-related awards 

♦ Recommendation by a person who met one of the criteria above 


This search resulted in a list of potential reviewers from both Oregon and non-Oregon institutions. 
Learning Outcomes Development participants included distinguished educators, as shown in Table 2. 


Table 2; Institutions Participating in Reviews 


Institution 

Review 1 

Review 2 

Oregon State University 

Pennsylvania State University - University Park 

X 

X 

University of Maryland - College Park 

X 

X 

University of Michigan - Ann Arbor 

X 

X 

University of Oregon 

X 


University of Portland 
University of Southern California 

X 

X 

Review 1 : 25 invited, 7 accepted, 5 compieted 
Review 2: 7 invited, 4 accepted, 4 compieted 


Study Recruitment and Participation 

This study included a variety of participant groups, including consultant reviewers, institutional liaisons, 
instructors, and paid course raters. 

Institutions 

Community colleges and Oregon University System (OUS) institutions with undergraduate programs 
were eligible for participation. This study sought to evaluate all core pre-engineering courses in the 
institutions shown in Table 3. 


Table 3; Oregon Community Colleges and OUS Institutions 


Community Colleges (17) 

OUS Institutions (7) 

Blue Mountain Community College 

Eastern Oregon University 

Central Oregon Community College 

Oregon Institute of Technology* 

Chemeketa Community College 

Oregon State University 

Clackamas Community College 

Portland State University 

Clatsop Community College 

Southern Oregon University 

Columbia Gorge Community College 

University of Oregon 

Klamath Community College 

Western Oregon University 

Lane Community College 
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Linn-Benton Community College 


Mt Hood Community College 


Oregon Coast Community College 


Portland Community College 


Rogue Community College 


Southwestern Oregon Community College 


Tillamook Bay Community College 


Treasure Valley Community College 


Umpqua Community College 



*Oregon Institute of Technology includes both the Klamath Falls and Portland campuses. 


Liaisons 

A liaison for each institution was recruited to work with EPIC during the study. The primary role of the 
liaisons was to be an informed participant at the institution who could confirm and correct the preliminary 
engineering course and instructor lists. Liaisons were recruited from the 24 community colleges and OUS 
institutions in Oregon. 

As shown in Table 3 above, 17 community colleges and seven OUS sites participated in the study. Each 
of the campuses for the Oregon Institute of Technology was treated as a separate institution for purposes 
of tracking pre-engineering course submissions and reviews, bringing the institutional count to 25. On 
August 7, 2006, letters were sent to deans, associate deans, engineering program directors, and 
community college presidents explaining the project and requesting their recommendations for liaisons at 
their institutions. By August 24* many of the institutions had identified a contact. Oregon Coast 
Community College declined to participate, as they have no courses required for pre-engineering and 
would counsel any student interested in engineering to transfer after taking basic courses the first year. 
The campus offered no calculus courses or 200-level science courses that would apply to this study. 

Of the final 24 liaisons, 12 were administrators (deans, department chairs, pre-engineering directors etc.) 
that were not currently teaching 100-200 level courses. Twelve of the liaisons were professors currently 
teaching 100- and 200-level courses, although many of them also served as engineering transfer 
coordinators, department chairs, or similar roles. Erom August through early October liaisons confirmed 
and corrected the course and instructor lists that had been developed by EPIC from institutional course 
catalogs and website information. 

Instructors 

On October 23, 2006, all instructors identified by liaisons (approximately 200 instructors) received an 
email explaining the goals of the Oregon Pre-engineering and Applied Sciences Study and inviting them 
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to participate. They were given access to a secure online course review tool containing the learning 
outcomes. EPIC researchers informed instructors that the purpose of reviewing courses was to develop a 
common list of learning outcomes for pre-engineering studies in both two- and four-year Oregon 
institutions. Courses relevant to the study were 100- and 200-level courses that are core requirements for 
a pre-engineering or engineering program. Instructors were asked to submit and review any course 
considered a pre-requisite to an engineering program at their own institutions or for transfer to another 
Oregon engineering program. Mathematics, science, and computer science courses were included if 
required for the engineering program, but not if the course was considered solely a general education 
requirement. 

In addition, instructors electronically submitted course documents to supplement their course reviews. 
Course documents could include a syllabus, major assignments, exams, or any other document that could 
provide evidence of course content. Submitting course documents was optional, but encouraged. 

Raters 

Raters, who were paid content experts, were trained to evaluate course documents submitted by 
instructors for evidence of the Oregon Pre-engineering Learning Outcomes. The purpose of the external 
review process was to provide an objective rating of the course to identify whether evidence of each 
learning outcome statement existed within the documents submitted for the course. Rater course review 
results were used in combination with instructor course reviews to identify the knowledge and skills 
necessary in the courses presented. 

Eleven professors, representing eleven different institutions of higher education across the United States, 
were trained to review course documents for this process. All of these external raters have experience 
teaching undergraduate courses in the subjects for which they were asked to conduct reviews. 

Data Collection 

Overview 

The Oregon Pre-engineering Learning Outcomes Project applied a modified Delphi process and included 
several data collection processes. The data collection and analysis began with development of learning 
outcomes. Once the learning outcomes were identified, a review process was implemented to pilot the 
learning outcomes and establish alignment between the draft outcomes and entry-level courses in Oregon 
engineering programs. Telephone interviews were conducted with a sample of faculty members to gain 
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additional information on the learning outcomes and to solicit their candid opinions about pre-engineering 
studies in Oregon. 

Learning Outcomes Development 

The learning outcomes were developed through several steps that included research on best practices in 
engineering education, multiple reviews and edits of draft outcomes by distinguished engineering 
educators, and a final review by OP AS stakeholders. 

1. Online Research of Existing Pre-engineering Learning Outcomes 

EPIC developed learning outcomes from research on best practice in engineering education. Over 100 
websites of national organizations, state education departments, and other engineering-related educational 
organizations were reviewed for standards for engineering content knowledge, key cognitive strategies, 
and teaching practices. Primary sources are listed below. 

♦ Center for Mathematics, Science, and Technology (CeMaST) 

♦ Knowledge and Skills for University Success (Physics and Chemistry) 

♦ Project 2061 (American Association for the Advancement of Science) 

♦ Center for Engineering Learning and Teaching (CELT) 

♦ Connections Project (Illinois State University) 

♦ Massachusetts Department of Education Standards 

♦ New York State Standards 

♦ Oregon Skill Sets (Oregon Department of Education) 

Standards were synthesized, combining like statements and including only general engineering 
expectations. Eield-specific expectations were not included because this study focused on the pre- 
engineering curriculum that prepares students to advance to any field of engineering. This research 
culminated in a draft list of 14 topical areas and 88 performance expectations across three sections: 1) 
Content Expectations, 2) Key Cognitive Strategies, and 3) Teaching Practices. 

2. Review One 

The initial review of the learning outcomes was available online from August 21, 2006, through 
September 4, 2006. Seven distinguished faculty from Oregon and non-Oregon institutions were recruited 
as unpaid reviewers, and five completed review one (see Table 2 above). Reviewers had the option to 
edit, delete, or add to the 88 expectations. Eor each expectation, reviewers were asked to indicate the level 
of importance for preparing students for engineering programs. Ratings included: 

4 - Most important for preparing students for engineering programs 
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3 - More important for preparing students for engineering programs 
2 - Less important for preparing students for engineering programs 
1 - Least important for preparing students for engineering programs 

Reviewers were asked to base the importance rating on their overall opinion of how important they 
believed each statement was for preparing students to move on to professional engineering programs. In 
order to reflect best practice in college readiness, reviewers also eliminated statements that were not the 
equivalent of at least a first-year college course. 

The rating was used to reduce the number of outcomes included in the final list. Statements with an 
average rating of less than 2.5 were not included in Review Two, unless at least one rater had assigned a 
rating of most important. Additionally, statements were included in Review Two when a concept 
appeared in multiple best practice sources and there was not clear reviewer agreement on the importance 
rating. 

3. Review Two 

Five reviewers agreed to participate in the second review and were sent a token of appreciation, a “Made 
In Oregon” gift basket, for their efforts. Comments and ratings from the first review were synthesized to 
create Review Two, resulting in 3 sections, 12 topics, and 73 expectations. Review Two was available 
online from September 7, 2006, through September 13, 2006. The same criteria for synthesis were applied 
here as were used in the first review, resulting in a final list of 3 sections, 10 topics, and 63 expectations. 

4. Final Learning Outcomes 

The synthesis and review process eliminated 25 non-essential or remedial-level learning outcomes. EPIC 
shared this milestone with OP AS stakeholders in order to receive feedback on the outcome statements 
before beginning the Oregon course review process. Feedback indicated that the outcomes should better 
reflect the strong connections between engineering and science, mathematics, and information 
technology. 

EPIC reorganized the learning outcomes, eliminating the topical headings and replacing them with 
content-based headers. With permission from the College Board, EPIC added content-specific 
performance expectations based on best practice research in high school Advanced Placement® courses in 
science, including, biology, chemistry, and physics. Mathematics expectations were further developed by 
a content expert and consultant experienced in standards development for college readiness. Additionally, 
computer science expectations were developed through online research of current college expectations 
and teacher training for information technology education. 
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The inclusion of topical areas in each of those areas resulted in the addition of 83 content expectations, as 
listed in Table 4. 


Table 4; Number of Learning Outcomes by Section and T opical Area 


Section 

Topical Area 

Number of Learning Outcomes 

Content Expectations 


108 


General Engineering 

25 


Mathematics 

19 


Biology 

14 


Chemistry 

15 


Physics 

15 


Computer Science 

20 

Key Cognitive Strategies 


31 

Teaching Practices 


12 

TOTAL NUMBER OF LEARNiNG OUTCOMES 

151 


The complete list of Oregon Pre-engineering Learning Outcomes is presented in Appendix A, and results 
from the instructor and rater questions are provided in Appendices B through F. 

Oregon pre-engineering course aiignment to iearning outcomes 

Once the draft learning outcomes were developed, EPIC researchers worked with liaisons at each Oregon 
community college and OUS institution to identify instructors of pre-engineering courses. Instructors 
were asked to review each course they had taught in the last two years, were currently teaching, or would 
be teaching in the 2006-2007 academic year. Because the online tool was extensive and required multiple 
decisions, EPIC allowed an extended period for instructor course reviews. Raters then reviewed submitted 
course documents to confirm instructor reviews. 

Initial pre-engineering course and instructor identification 

EPIC researchers reviewed course catalogues and department websites to identify over 700 requiredlOO- 
and 200-level courses in engineering programs at the 24 Oregon institutions. From late August 2006 to 
October 2006, liaisons used an online tool to confirm the course list and to submit contact information for 
instructors of each course. 

Instructors were given the following guidelines for refining the course lists: 

■ Algebra courses were not included, as they were viewed as prerequisites to postsecondary 
study. 

■ Engineering tech courses (courses required for one- and two-year certificates) were not 
included, as only courses that contributed to a four-year degree from an engineering program 
were eligible for review. 
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■ Geography, geology, economics, English, technical report writing, and other courses were not 
included although they may be required for general education requirements or electives. 

■ Introduction to Windows and other low-level computing courses were not included, as well 
as computer gaming courses that were non-major courses. 

■ Labs were not included as separate courses, but instructors were told to include the lab 
content in the associated course review. 

■ Courses titled “Special Topics” at schools were not included. 

■ Spring courses without an assigned instructor were only included if a past instructor could be 
reached to complete the review. 

■ If an institution submitted contact information for more than one instructor for the same 
course, only one was contacted. If a professor was listed for multiple courses, EPIC staff 
made every effort to spread the course review load across other available professors. 

■ Liaisons could participate as both an instructor and liaison for their institutions. 

Liaison’s final confirmation of courses and instructors resulted in a list of just over 600 coursers across all 
participating institutions. Instructors received an email invitation to participate in the Oregon Pre- 
engineering and Applied Sciences Study in mid-October 2006. Liaisons could monitor instructor progress 
online and were contacted by EPIC staff if there were concerns about participation. 

Instructor online course review tool 

Instructors logged into a web-based data collection tool to complete a course profile of each course they 
taught, review the course for evidence of the Oregon Pre-engineering Learning Outcomes, and submit 
course documents. 

Lor each performance expectation, or learning outcome, instructors were prompted to respond to four 
questions: 

1 . How important is each performance expectation in my course? 

2 . What do students need to know when they enter my class? 

3. What do I observe students knowing when they enter my class? 

4. How important is the performance expectation in preparing students for further study in 
engineering programs? 

Although responses to each question were optional, instructors were strongly encouraged to respond to all 
expectations listed in the engineering strand of the content section, key cognitive strategies, and teaching 
practices. Instructors were also asked to offer any general comments about the learning outcomes and 
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answer open-ended questions about student preparation for postsecondary study in engineering programs. 
In addition to completing the online course review, faculty had the option to upload accompanying course 
documents. This could include syllabus, major assignments, exams, or other supporting documents. 

In late November 2006, liaisons and instructors requested course review extensions. Several extensions 
were given to accommodate teaching and grading demands as well as a community college curriculum 
review that could produce outcomes that would change course content. Participation was slow but steady 
and required persistent communication between EPIC staff, liaisons, and instructors. 

While feedback regarding the goals of the study was overwhelmingly positive, there were several issues 
identified by instructors during the course review process. Key issues included: 

♦ Survey is too long and complex — making it difficult to complete. 

♦ Instructors of some mathematics or science courses found it difficult to identify 
engineering-specific outcomes. 

♦ In several specific cases one instructor taught 5-12 courses included in the study. 

To assist instructors a few adjustments were made to course review submissions: 

♦ If an instructor taught a series course (e.g.. Physics 21 1, 212, 213), one review could be 
submitted for all three courses. 

♦ When instructors declined to participate in the four questions of the course review 
process, they were encouraged to email course documents to EPIC. 

The online instructor course review tool was available through January 2007. As listed in Table 5 below, 
just over half of identified instructors of pre-engineering courses participated in the instructor course 
reviews, including instructors who chose to email syllabi in lieu of completing the course reviews. 

Table 5; Completed Instructor Course Reviews by Course Type 


Course Type 

Total 

Expected 

Total 

Completed 

Percent 

Complete 

Engineering 

274 

135 

49.3% 

Mathematics 

109 

63 

57.8% 

Biology 

6 

3 

50.0% 

Chemistry 

60 

28 

46.7% 

Physics 

76 

40 

52.6% 

Computer Science 

83 

45 

54.2% 

Totals 

608 

314 

51.6% 
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Consultant rater review of submitted course documents 

Consultant raters provided external validation of what the course documents reflect about the courses. 
EPIC staff trained paid raters on use of the tool and the criteria for establishing evidence of learning 
outcomes through the course documents. Using an online rating tool to view PDF files of the submitted 
course documents, raters indicated if evidence was found for each learning outcome. Between March and 
May 2007, raters reviewed documents submitted for 216 courses. Table 6 shows the number of courses 
per content area for which supporting documents were submitted. 

Table 6; Number of Courses by Content Area with Documents Submitted 

for External Review 


Content Area 

Number of 
Courses 

Engineering 

112 

Mathematics 

43 

Biology 

4 

Chemistry 

19 

Physics 

17 

Computer Science 

21 

Total 

216 


Instructor interviews 

Instructor interviews were added to the study to supplement information reported via the online tool and 
course documents. Primarily instructors of courses under the engineering content area were invited to 
participate in the interviews, particularly given the feedback from mathematics and science instructors 
regarding their difficulty in identifying engineering-specific learning outcomes for non-engineering 
content areas. An additional effort was undertaken to include instructors who were unable to participate in 
the online course review process. 

Under these general criteria for participation, 52 of 71 (73%) invited engineering instructors participated 
in short phone interviews. This group included instructors who both participated and did not participate in 
the online course review and document submission process. As shown in Table 7, about two-thirds of 
interviewees taught courses at OUS institutions. 


Table 7; Instructor Interviews Participation by Institution T ype 


Interview Participation 

Community College 
Participants 

OUS Institution 

Total Number of 
Participants 

Instructors invited 

29 (41%) 

42 (59%) 

71 

instructors participating 

1 9 (36%) 

33 (64%) 

52 
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Interview questions 

Telephone interviews were conducted over a three- week period beginning in mid-May 2007 and 
generally lasted 10-15 minutes. Selected-response items were included to clarify responses from the 
instructor and reviewer process. Of particular consideration was further identification of the most 
important learning outcomes for pre-engineering. Responses to open-ended questions were also solicited 
to record instructor impressions of student preparation for postsecondary study, including: 

1 . What could students do in high school to be better prepared to succeed in entry-level 
engineering courses? 

2 . How can programs in the high school be improved so students enter courses ready to 
succeed? 

3. How do students demonstrate thinking skills (critical thinking, analytic thinking, coping 
with ambiguous tasks, interpretation, precision & accuracy) in their classes? 

4. How do students who don’t demonstrate thinking skills perform differently in their 
classes? 

5. Based on their experience and observation, what are the major reasons students do not 
continue beyond entry-level courses in engineering? 

Limitations of the Research Design and Methodology 

Although this study provides a strong foundation for the development of pre-engineering learning 
outcomes, there are limitations to the methodology and the process applied over the course of the study. 
The draft of learning outcomes developed by reviewers resulted in a strong set of general engineering 
learning expectations. However, feedback from engineering educators and industry experts resulted in the 
addition of content-specific standards. While the inclusion of content areas increased the explicit links 
between general engineering, science, mathematics, and computer science, it also increased the length of 
the reviews. Instructor fatigue resulted in fewer course review completions and course document 
submissions. EPIC researchers provided flexibility to instructors to reduce the amount of time required 
for online reviews. This also resulted in partial responses to the four questions when instructors chose to 
submit responses for only select outcomes. Instructors also chose to upload or email only their course 
documents, reducing the ability to provide external validation of instructor self reviews. Refining and 
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reducing the number of standards may increase transparency and successful adoption of the learning 
outcomes. 

Biology instructors submitted no course reviews, and only one course document was received for a 
biology course. Due to virtually no biology-specific instructor participation, the biology learning 
outcomes are not included in the following results and discussion. Biology learning outcomes, as 
developed by the reviewers and best practice research by EPIC, are provided in Appendix A. 


Oregon Pre-engineering Learning Outcomes Study 


- 14 - 


Findings & Interpretation 


Overview 

The draft list of Oregon Pre-engineering Learning Outcomes was developed to identify expectations for 
both high school students entering pre-engineering programs and postsecondary students preparing for 
upper-division engineering studies. A review of 100- and 200-level engineering courses in Oregon 
postsecondary institutions provided a baseline understanding of what is currently taught in the first two 
years of engineering courses. To assist in this effort, instructors from Oregon community colleges and 
universities reviewed over 300 pre-engineering courses in which they had teaching experience. The 
learning outcomes cover content knowledge, key cognitive strategies, and teaching practices. For each 
performance expectation in the content knowledge and key cognitive strategies sections, instructors 
responded to four questions: 

1 . What do students need to know to enter my class? 

2. What do I observe that students know? 

3 . How important is this for students to successfully complete this course? 

4 . How important is the learning outcome in preparing students for further engineering 
studies? 

Only the last two questions were included for the teaching practices section of the online course review. A 
synthesis of the reviews provides several general findings on the preparation and expectations of pre- 
engineering students. 

♦ Virtually all pre-engineering courses require more than a basic understanding of course 
content. 

♦ Instructors generally find that students enter their courses with adequate knowledge. 

♦ Two-thirds of learning outcomes are more or most important for the success in the first 
two years of engineering programs. 

♦ Almost 40% of the learning outcomes are more or most important for successful entry 
into upper-division engineering courses. Responses regarding future studies specifically 
identified finding all of the teaching practices learning outcomes as important for study 
beyond the first two years of engineering programs. 

Overall, there was little change in the instructors’ ratings of the importance of each of the learning 
outcomes for their course and for ongoing engineering study. This may indicate that the learning 
outcomes reflect college-level standards and are foundational for general engineering studies. Finally, the 
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overall importance of many of the learning outcomes, across content-specific expectations, may also 
reflect the cross-disciplinary nature of pre-engineering programs. Additional findings by content area are 
provided in the next sections of this report. 

Findings by Content Area 

The following five content areas are discussed below: general engineering, mathematics, chemistry, 
physics, and computer science. 

General Engineering 

Instructors submitted reviews for engineering courses from Oregon postsecondary institutions. Course 
reviews represent an even distribution between community colleges and OUS institutions, with Oregon 
Institute of Technology (OIT) representing one-half of the submitted OUS courses. 

Learning outcome importance 

Engineering instructors identified all learning outcomes for engineering content and key cognitive 
strategies to be important for success in both entry-level courses and advanced engineering coursework. 
Among the 53 instructors interviewed, the importance of key cognitive strategies was further evidenced 
by over three-quarters of interviewed instructors identifying student thinking skills as most important to 
success in their courses. Teaching practices were identified as having a greater level of important overall 
than course content and key cognitive strategies. 

With regard to specific learning outcomes, GE023, Convert between different unit and measurement 
systems, stood out as the most important learning outcome for success in the courses. The following 
outcomes were relatively less important: 

GE002: Understand that scientific knowledge used in technology is not a replacement for 
the trial-and-error method of technology; rather, it provides a means of selecting what 
trial to undertake next and thus contributes to the efficiency and effectiveness of the trial- 
and-error process. 

GE003: Know that technology usually affects society more directly than science because 
it solves practical problems and serves human needs ( and may create new problems and 
needs). In contrast, science affects society mainly by stimulating and satisfying people's 
curiosity and occasionally by enlarging or challenging their views of what the world is 
like. 
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GE015: Understand that as the number of parts of a system increases, the number of 
possible interactions between pairs of parts increases much more rapidly. 

Similarly, relative to the other learning outcomes, GEO 1 8 (below) was identified most important for 
preparing students for further study in engineering: 

Understand mathematics and become mathematically confident by communicating and 
reasoning mathematically, by applying mathematics in real-world settings, and by 
solving problems through the integrated study of number systems, geometry, algebra, 
trigonometry, differential equations, data analysis, and probability, 

and GE003 (below) as least important: 

Know that technology usually affects society more directly than science because it solves 
practical problems and serves human needs ( and may create new problems and needs ). 

In contrast, science affects society mainly by stimulating and satisfying people's curiosity 
and occasionally by enlarging or challenging their views of what the world is like. 

With key cognitive strategies, the following learning outcomes were identified as skills that are key to 
course success: 

KCS18: Make up and write out simple algorithms for solving problems that take multiple 
steps. 

KCS25: Use tables, charts, and graphs in making arguments and claims in oral and 
written presentations. 

KCS26: Check graphs to see that they do not misrepresent results by using inappropriate 
scales or by failing to specify the axes clearly. 

Learning outcomes KCS12: Demonstrate problem-solving skills by developing a process for a solution 
using relationships between words, diagrams, graphs, and mathematical representations and KCS13: The 
student can plan a solution for a given problem by determining the steps necessary to solve, verify the 
solution, and evaluate whether the proposed solution is reasonable were noted as most important for 
preparing students for further study. However, similar to the course content, very little evidence of key 
cognitive strategies learning outcomes was found among the course documents submitted by instructors, 
with exception to five learning outcomes (namely KCS10-KCS15) for which evidence was 
overwhelmingly found. This suggests that although the key cognitive strategies and thinking skills are 
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crucial to course success and preparing students for further study, not much is directly addressed in the 
entry-level courses. 

Teaching practices were found to be most important overall with learning outcome TP004: Helping and 
encouraging students to develop their abilities standing out as a key practice for both helping students 
succeed in the courses and for preparing students for further study. Interestingly, discussion of these 
teaching practices was found to be highly evident within the course documents, suggesting that instructors 
recognize the importance of these practices and convey their methods openly to students. 

Observations and expectations 

On average, roughly one-third of the instructors expressed that an engineering learning outcome was not 
taught in his/her course. However, for those who did include the specific learning outcomes, instructors 
identified that they expect college students to enter their entry-level engineering courses with only a basic 
understanding of the course material. Instructors further confirmed that they have observed that generally 
students do indeed enter their classes with adequate knowledge, thereby meeting the expectation. The 
same pattern was observed for the key cognitive strategies learning outcomes. 

The expectation of students only needing a basic understanding of the material implies that instructors 
plan to build upon this foundational knowledge and will therefore teach this material in their courses. 
However, this was not found to necessarily be the case. Although external reviewers did find some 
evidence of the learning outcomes being covered among the engineering courses, little evidence was 
found overall. Outside of the material identified as not being included in the course, there are a few 
possible explanations for this. It is possible that the course documents provided by the instructors were 
not the most descriptive or detailed documents that could have been provided. Or, it is possible that 
documents themselves should have been sufficient, but the detail provided that would identify the 
learning outcomes in the documents was insufficient or poorly represented. 

Preparation and success 

High school was identified as playing a significant role in a student’s preparedness for engineering 
courses in college; however the level of its significance varies among those interviewed. Overall, 50% of 
the interviewees observed that of the students who seem particularly well prepared to succeed in the 
instructor’s engineering courses, a great deal of their preparation can be attributed to their high school 
program of study. However, when broken down by the institution at which the instructor teaches, the 
majority of university instructors agree with this statement, whereas only 37% of community college 
instructors agree with this, the others observing that students’ high school preparedness attributes to their 
success to a lesser degree. 
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According to the 53 instructors interviewed, actions that high school students can take to better prepare 
themselves for college-level engineering courses are to: 

♦ Take more mathematics courses. 

♦ Take responsibility for their own learning. 

♦ Take more science courses, particularly physics or “hands-on” science. 

♦ Improve writing skills. 

♦ Seek challenges and complex problem solving. 

Students must increase their mathematics proficiency, particularly with regard to algebra. It was noted 
that drafting classes should also be taken when available as they help to fine-tune problem-solving skills 
by requiring students to apply their algebra knowledge. Exposure to calculus would also be helpful. Most 
importantly, it was expressed that students need to feel comfortable and confident with mathematics and 
be able to think beyond individual problems to real-world applications. 

Instructors also expressed that students can better prepare themselves in high school by taking 
responsibility for their learning by being self-disciplined enough to look beyond graduation requirements 
and take the additional courses they need (such as additional mathematics, drafting and physics) to 
prepare for engineering. Additionally, students need to be self-disciplined in their study habits to perform 
well in these courses. 

Although taking more science courses in general would be helpful, physics was identified specifically and 
repeatedly as a key discipline for preparing students for engineering. It should be noted that some 
instructors further clarified that the physics course taken does not need to be calculus based. 

When asked how high school programs of study can be improved so that students are better prepared for 
college engineering courses, the instructors stated that high schools should improve the way in which 
mathematics is currently being taught. More specifically, students need to better understand how to apply 
their mathematics skills across varying situations. Mathematics should be more highly integrated into 
science, drafting, and other course subjects to assist in understanding the applications of mathematics. 
Students should be taught how to approach problems and become more aware of the process involved in 
solving problems as opposed to simply being focused on whether the answer is correct. It was also noted 
that better guidance from high school counselors and instructors would be beneficial to students by 
steering them towards more appropriate courses and encouraging their studies. 
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Furthermore, it was the observation of engineering instructors that students do not continue beyond entry- 
level courses in engineering primarily because they find the engineering program simply too difficult. 
Students are either unprepared for rigorous coursework, particularly with regard to mathematics, or, in 
many cases, either lack the self-discipline or are simply unwilling to apply themselves appropriately. 
However, instructors also pointed out that while the many students may be ill prepared, they are not 
necessarily unqualified. Many of the students who do not continue on with engineering are perfectly 
capable of doing the work, but it is possible that they discontinue the program because they do not have 
realistic expectations and end up being intimidated by the coursework and the amount of time their 
studies require. 

Finally, postsecondary instructors advise high school educators and career counselors to work with local 
community colleges and industry to provide opportunities for students to see what engineers actually 
do — students need opportunities to see the exciting aspects of engineering and to connect with people that 
are passionate about what they do. 

Mathematics 

Data were collected from 53 mathematics courses from among Oregon postsecondary institutions. All 
mathematics courses were identified as core courses required for engineering majors. These courses 
represented both two- and four-year institutions with about two-thirds from community colleges. 

Learning outcome importance 

Instructors identified all mathematics learning outcomes to be important for success in their entry-level 
courses, but less important for further engineering coursework. The key cognitive strategies learning 
outcomes, however, were identified as important for success in the courses as well as further engineering 
coursework. Interestingly, teaching practices were identified as having a greater level of importance 
overall than course content and key cognitive strategies for both success in current and future engineering 
coursework. 

With regard to specific learning outcomes, MA006: Have knowledge of gradient, curl, Jacobian, and 
Laplacian stood out as the most important learning outcome for success in the courses and outcome 
MA016: Use Newtonians method for approximation was the least in relative importance. Similarly, 
relative to the other learning outcomes, the following three were identified as key elements for preparing 
students for further study in engineering: 

MAOOl: Perform operations on matrices and use them to solve systems of linear equation 
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MA008: Use differentiation to optimize functions of one variable 


MA009: Understand implicit differentiation and its use for related rates problems 

Learning outcome MA019: Use partial fractions and trigonometric substitutions for integration was rated 
least important. 

With key cognitive strategies, learning outcome KCS27: Describe, in words, both written and orally, an 
experiment or derivation he or she has made to an audience of non-experts in a manner that avoids 
jargon and follows a clear logical path between question, method, solution, and impact was identified as 
a key element for course success and learning outcomes. KCS13: The student can plan a solution for a 
given problem by determining the steps necessary to solve, verify the solution, and evaluate whether the 
proposed solution is reasonable and KCS22: Determine whether the mathematical solutions are 
reasonable by reviewing the process and checking against typical values from experience of the everyday 
world were noted as most important for preparing students for further study. However, similar to the 
course content, very little evidence of key cognitive strategies learning outcomes was found among the 
course documents submitted by instructors, with exception to three learning outcomes, namely KCS14: 
The student can determine known and unknown quantities that are relevant to a given problem and 
KCS29: Use and correctly interpret relational terms such as if . . . then . . . and, or, sufficient, necessary, 
some, every, not, correlates with, and causes for which evidence was overwhelmingly found. This 
suggests that although the key cognitive strategies and thinking skills are crucial to course success and 
preparing students for further study, not much is directly addressed in the entry-level courses. 

Teaching practices were found to be most important overall with learning outcome TP004 standing out as 
a key practice for helping students succeed in the courses. Although still considered important, learning 
outcomes TP009: Connecting academic ideas with the way a professional in the field thinks and explains 
the practical issues of the topics covered and TP012: Raising questions consciously that cause students to 
consider the quality of evidence were identified as less critical overall for success in current coursework 
and for helping prepare students for further study. Interestingly, more evidence of these teaching practices 
was found within the course documents than of course content or key cognitive strategies, suggesting that 
instructors recognize the importance of these practices and convey their methods more openly to students. 

Observations and expectations 

On average, roughly one-half of the instructors expressed that a mathematics learning outcome was not 
taught in his or her course. However, for those who did include the specific learning outcomes, instructors 
identified that they expect college students to enter their entry-level mathematics courses with only a 
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basic understanding of the course material. Instructors further confirmed they have observed that students 
generally do enter their classes with adequate knowledge, thereby meeting the expectation. The same 
pattern was observed for the key cognitive strategies learning outcomes. However, instructors expressed 
that they expect students to have a more thorough understanding of learning outcomes KCS22: Determine 
whether the mathematical solutions are reasonable by reviewing the process and checking against typical 
values from experience of the everyday world and KCS31: Write clear, step-by-step instructions for 
conducting investigations, operating something, or following a procedure before they enter the class and 
that students’ knowledge and practice of learning outcomes KCSl 1 : Apply knowledge and thinking skills 
to address real life problems and make informed decisions and KCS17: Make up and write out simple 
algorithms for solving problems that take multiple steps has been observed to be less than adequate in 
general. 

Chemistry 

Data were collected from 19 chemistry courses from among Oregon postsecondary institutions. All 
chemistry courses were identified as core courses required for engineering majors. These courses 
represented both two- and four-year institutions with about two-thirds from community colleges. 

Learning outcome importance 

Instructors identified all chemistry learning outcomes to be most important for success in their entry-level 
courses and important as well for further engineering coursework, although to a slightly lesser degree. 
Following a similar pattern, the key cognitive strategies learning outcomes were identified as most 
important for success in the courses and nearly as important for further engineering coursework. 

However, teaching practices were identified as having a greater level of importance overall than course 
content and key cognitive strategies for both success in current and particularly for future engineering 
coursework. 

With regard to specific learning outcomes, CH006: Know the basic subatomic constituents of matter (i.e., 
proton, neutron, electron) stood out as a key learning outcome for success in the courses. Learning 
outcome CH013: Experimentally measure and calculate limiting reactant and percent yield was 
identified as the least in relative importance with regard to the courses, but was identified as a key 
element for preparing students for further study in engineering along with CH012: Use stoichiometry to 
measure or calculate the relationship among mass, moles, and number of particles of reactants and 
products in a balanced chemical equation and CHOI 5: Links chemical concepts to personal observations 
and experiences and chemically related news items. Considerable evidence of the chemistry learning 
outcomes was found within the course documents submitted by instructors. This is a good indication that 
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chemistry instructors are doing a good job of teaching material that is most important and critical for 
engineering students. 

For key cognitive strategies, learning outcomes KCS28: Participate in group discussions on scientific 
topics by restating or summarizing accurately what others have said, asking for clarification or 
elaboration, and expressing alternative positions and KCS31: Write clear, step-by-step instructions for 
conducting investigations, operating something, or following a procedure were identified as key elements 
for course success and learning outcomes. The outcomes listed below were noted as most important for 
preparing students for further study. 

KCSOl: Know why curiosity, honesty, openness, and skepticism are so highly regarded in 
scientific research and how they are incorporated into the way engineers carry out science. 

KCS04: Know that there may be ethical issues, such as ignoring contradicting data and selecting 
only supporting data, when considering claims. 

KCS13: The student can plan a solution for a given problem by determining the steps necessary 
to solve, verify the solution, and evaluate whether the proposed solution is reasonable. 

Teaching practices were found to be most important overall with learning outcome TP004: Helping and 
encouraging students to develop their abilities standing out as important. Interestingly, considerable 
evidence of these teaching practices was found within the course documents, suggesting that instructors 
recognize the importance of these practices and convey their methods openly to students. 

Observations and expectations 

Very few of the instructors who participated in this study expressed that a chemistry learning outcome 
was not taught in his/her course. Instructors identified that they expect students to enter their entry-level 
chemistry courses with a more than a basic understanding of the chemistry course material, particularly 
more with regard to learning component CH006: Know the basic subatomic constituents of matter (i.e., 
proton, neutron, electron). Instructors were nearly evenly split between whether they observe students’ 
knowledge to be adequate with regard to the expected level of knowledge. The same pattern was observed 
for the key cognitive strategies learning outcomes. However, instructors expressed that they expect 
students to have a more thorough understanding of learning outcomes KCS02: The student regards 
science as an essential way of knowing and learning engineering professions and KCS27: Describe, in 
words, both written and orally, an experiment or derivation he or she has made to an audience of non- 
experts in a manner that avoids jargon and follows a clear logical path between question, method, 
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solution, and impact and they observe that students’ knowledge is indeed adequate for this raised 
expectation. 

Physics 

Data were collected from 31 physics courses from among Oregon postsecondary institutions. All physics 
courses were identified as core courses required for engineering majors. These courses represented both 
two- and four-year institutions with about two-thirds from community colleges. 

Learning outcome importance 

Instructors identified all physics learning outcomes to be highly important for success in their entry-level 
courses, but slightly less important for further engineering coursework. The key cognitive strategies and 
teaching practices learning outcomes followed an opposite pattern, identified as important for physics 
courses and even more important for further engineering coursework. Interestingly however, both were 
identified as less important overall than the physics content. 

With regard to specific learning outcomes, PH013: Describe, using words, diagrams and equations, the 
forces on and motion of charges in given electric and magnetic fields stood out as the most important 
content learning outcome for success in physics courses. Similarly, relative to the other learning 
outcomes, the following were identified as key elements for preparing students for further study in 
engineering. 

PH002: Describe, using words and a free-body diagram, the forces acting on a given object or a 
system of interacting objects. 

PH003: Describe, using words, graphs, or equations the motion of a given object or system of 
interacting objects that results from forces acting upon the object or objects. 

PH008: Describe, using words, graphs or equations, the change of linear momentum of a given 
system under different conditions, including before and after an interaction, and any transfer of 
linear momentum into or out of the system, using the principle of the conservation of linear 
momentum. 

The physics learning outcomes were found to be highly evident throughout the course documents, their 
presence supporting the importance of these standards. 

For key cognitive strategies, learning outcome KCS12: Demonstrate problem-solving skills by developing 
a process for a solution using relationships between words, diagrams, graphs, and mathematical 
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representations was identified as the most important element for course success followed closely by 
outcomes KCS13: The student can plan a solution for a given problem by determining the steps necessary 
to solve, verify the solution, and evaluate whether the proposed solution is reasonable and KCS14; The 
student can determine known and unknown quantities that are relevant to a given problem. 

Similarly, learning outcome KCS03; Insist that the critical assumptions behind any line of reasoning be 
made explicit so that the validity of the position being taken-whether one's own or that of others-can be 
judged was noted as a key element for preparing students for further study. Conversely, the following 
learning outcomes stood out as the least important elements for course success as well as further 
engineering studies. 

KCS05: Know what is expected of a professional, such as knowing a professional code of ethics. 

KCS09: Understand that scientists use methods and knowledge drawn from more than one 
scientific discipline as they solve problems. 

KCS21: Be able to estimate how likely it is that some event of interest might have occurred just by 
chance. 

Evidence of the key cognitive strategies outcomes was found in the course documents, but not 
overwhelmingly so. However, the degree of evidence found was proportional to the importance indicated, 
such that the learning outcomes identified as most important were more highly evident than the others, 
with the least important being least evident. This suggests that the importance of key cognitive strategies 
and thinking skills most crucial to course success and preparing students for further study are recognized 
and addressed appropriately in the entry-level courses. 

Teaching practices were found to be nearly as important as the physics content, with the following 
learning outcomes standing out as key practices for helping students succeed in the courses. 

TP004: Helping and encouraging students to develop their abilities. 

TP005: Working to develop students' good study habits, self-reliance and responsibility for 
learning. 

TP008: Developing goals and pace for the course that are appropriately challenging. 

Outcome TP004: Helping and encouraging students to develop their abilities was also identified as the 
most important practice for preparing students for further study in engineering. The teaching practice 
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learning outcomes were highly evident within the course documents, suggesting that instructors recognize 
the importance of these practices and convey their methods openly to students. 

Observations and expectations 

Very few of the instructors who participated in this study expressed that a physics learning outcome was 
not taught in the course. Instructors identified that they expect college students to enter their entry-level 
physics courses with less than a basic understanding of the course material. Instructors further confirmed 
that they have observed that generally students do indeed enter their classes with adequate knowledge, 
thereby meeting the expectation. The same pattern was observed for the key cognitive strategies learning 
outcomes. Instructors expressed that they expect students to have a stronger basic understanding of 
learning outcome KCS29 before they enter the class and have observed that students’ knowledge and 
practice of this skill has been generally observed to be adequate. 

Computer Science 

Data were collected from 31 computer science courses from among Oregon postsecondary institutions. 

All computer science courses were identified as core courses required for engineering majors. These 
courses were fairly evenly distributed between community colleges, universities, and OIT, with roughly 
one-third of the courses from each. 

Learning outcome importance 

Instructors identified all computer science learning outcomes to be important for success in their entry- 
level courses and equally important for further engineering coursework. Similarly, key cognitive 
strategies and teaching practices learning outcomes were identified as important for success in the courses 
and equally important for further engineering coursework as well. Interestingly however, teaching 
practices were identified as having a greater level of importance overall than computer science content 
and key cognitive strategies for success in both computer science courses and future engineering 
coursework. 

With regard to specific learning outcomes, CSOOl: Demonstrate knowledge of and skill regarding the 
syntax and semantics of a high level programming language, its control structures, and its basic data 
representations and CS015: Implement syntactically correct, logically accurate programs based on 
logical algorithms/pseudo code stood out as the most important content learning outcome for success in 
computer science courses. Similarly, learning outcome CSOOl was also identified as a key element for 
preparing students for further engineering coursework along with outcome CS004: Demonstrate 
knowledge of and skill regarding program correctness issues and practices (e.g., testing program results, 
test data design, loop invariants ). Despite their identified importance however, little evidence of the 
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computer science learning outcomes was found throughout the course documents submitted by 
instructors. 

For key cognitive strategies, learning outcomes KCS13: The student can plan a solution for a given 
problem by determining the steps necessary to solve, verify the solution, and evaluate whether the 
proposed solution is reasonable and KCS31: Write clear, step-by-step instructions for conducting 
investigations, operating something, or following a procedure were identified as key elements for success 
in computer science courses. Similarly, the following learning outcomes were noted as most important 
elements for preparing students for further engineering study. 

KCSll: Apply knowledge and thinking skills to address real life problems and make informed 
decisions. 

KCS13: The student can plan a solution for a given problem by determining the steps necessary 
to solve, verify the solution, and evaluate whether the proposed solution is reasonable. 

KCS14: The student can determine known and unknown quantities that are relevant to a given 
problem. 

Conversely, learning outcomes KCS06: Sample a wide range of scientific literature at a superficial level 
so as to build up an approximate framework for interpreting what is found on more detailed reading and 
KCS21: Be able to estimate how likely it is that some event of interest might have occurred just by chance 
stood out as the least important elements for course success as well as further engineering studies. 
Surprisingly, considerably little evidence of the key cognitive strategies learning outcomes was found 
within the course documents. This suggests that although the key cognitive strategies and thinking skills 
are crucial to course success and preparing students for further study, not much is directly addressed in 
the entry-level courses. 

Teaching practices were found to be relatively more important than the physics and key cognitive 
strategies learning outcomes with regard to computer science course and further engineering study 
success. Learning outcome TPOOl: Providing timely and suitable feedback to students stood out as key 
practice for helping students succeed in the courses as well as TP004: Helping and encouraging students 
to develop their abilities which was also identified as the most critical element for preparing students for 
further study in engineering. Although some evidence of the teaching practice learning outcomes was 
found within the course documents, little evidence was found overall suggesting that instructors may not 
openly convey their methods to students. 
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Observations and expectations 

Roughly one-quarter to one-half of the instructors who participated in this study expressed that a 
computer science learning outcome was not taught in the course. Instructors identified that they expect 
college students to enter their entry-level physics courses with less than a basic understanding of the 
course material. Instructors further confirmed that they have observed that generally students do indeed 
enter their classes with adequate knowledge, thereby meeting the expectation. The same pattern was 
observed for the key cognitive strategies learning outcomes. 
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Recommendations 


Implications of Findings 

The Oregon Pre-engineering Learning Outcomes Study is the first of its kind to examine learning 
expectations in the first two years of Oregon engineering courses. Refinement of existing national and 
state standards in engineering education and related content areas provide a foundation for the draft 
outcomes included in this report. As a starting place for standards development, the outcomes reflect a 
broad scope and general performance expectations. 

The outcomes are presented to ETIC as draft outcomes, as EPIC acknowledges that these are only the 
beginning of additional investigation. 

We recommend improving the draft outcomes through additional studies, such as: 

1 . Additional analysis of alignment between engineering programs at Oregon community 
colleges and OUS institutions.. 

2. A “best practices”-style study, including a deeper and broader review of course 
documents, to find exemplars of Oregon courses that already fully incorporate the 
learning outcomes. 

3. An investigation of vertical alignment between the three primary engineering schools 
(Oregon State, Oregon Institute of Technology, and Portland State) and other 
postsecondary institutions in Oregon. 

4. Eocused studies on each of the subject areas to illuminate the explicit connections 
between engineering, mathematics, science, and technology courses. 
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Appendix A: The Oregon Pre- engineering 

Learning Outcomes 


General Engineering 


ID 

Outcome 

GE001 

Know that mathematics, creativity, logic, and originality are all needed to improve technology. 

GE002 

Understand that scientific knowledge used in technology is not a replacement for the trial-and-error 
method of technology; rather, it provides a means of selecting what trial to undertake next and thus 
contributes to the efficiency and effectiveness of the trial-and-error process. 

GE003 

Know that technology usually affects society more directly than science because it solves practical 
problems and serves human needs (and may create new problems and needs). In contrast, 
science affects society mainly by stimulating and satisfying people's curiosity and occasionally by 
enlarging or challenging their views of what the world is like. 

GE004 

Understand that the professional practice of applying the engineering design process is used in the 
solution of problems and the advancement of society. 

GE005 

Identify and explain the steps of the engineering design process (i.e., identify a design problem, 
propose a solution, construct one or more prototypes and/or models, evaluate design, and revise 
design). 

GE006 

Communicate solutions by making an engineering presentation that includes a discussion of how 
the solution best meets the needs of the initial problem or opportunity. 

GE007 

Consider cost constraints and how the technology will be manufactured, operated, maintained, 
replaced, and disposed of and who will sell, operate, and take care of it. 

GE008 

Understand that problem solving begins with specifying system boundaries and subsystems, 
indicating its relation to other systems, and identifying what its input and its output are expected to 

bo. 

GE009 

Understand that the basic idea of mathematical modeling is to find a mathematical relationship that 
behaves in similar ways as the system or processes under investigation. 

GE010 

Interpret multi-view drawings (orthographic projections) and pictorial (isometric, oblique, 
perspective) drawings using various techniques. 

GE011 

Interpret plans, diagrams, and working drawings in the construction of prototypes or models. 

GE012 

Test the usefulness of a model by comparing its predictions to actual observations in the real world 
with consideration of alternative solutions. 

GE013 

Apply statistics, graphs, and equations as useful ways for depicting and analyzing patterns of 
change and drawing conclusions. 

GE014 

Recognize that statistical scatter results from uncertainties in measurements and specification 
ranges of quantities and therefore impacts these quantities along with all calculations using them. 

GE015 

Understand that as the number of parts of a system increases, the number of possible interactions 
between pairs of parts increases much more rapidly. 

GE016 

Understand scientific notation, the concept of a logarithm, logarithmic scales, and the use of units 
appropriate for the problem at hand. 

GE017 

Know that representing large numbers in terms of powers of ten makes it easier to think about 
them and to compare things that are greatly different. 

GE018 

Understand mathematics and become mathematically confident by communicating and reasoning 
mathematically, by applying mathematics in real-world settings, and by solving problems through 
the integrated study of number systems, geometry, algebra, trigonometry, differential equations, 
data analysis, and probability. 

GE019 

Make measurements or calculations to collect data and analyze the data using appropriate 
statistics such as mean, standard deviation, chi-square, linear regression and correlation. 

GE020 

Construct and use tables and graphs to interpret data sets. 

GE021 

Solve algebraic equations with several variables including equations with unspecified parameters. 

GE022 

Measure with accuracy and precision (length, volume, mass, temperature, time, etc.) 
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ID 


GE023 

GE024 

GE025 


Outcome 


Convert between different unit and measurement systems. 
Determine the correct number of significant figures. 

Determine percent error from experimental and accepted values. 


Note: All outcomes below that are marked with an asterisk are used with permission by the College Board 
AP® program. 


Chemistry 


ID 

Outcome 

CH001 

CH002 

CH003 

Use periodic trends to predict the chemical and physical properties of elements. 

Describe and use the periodic trends in electronegativity and metallic character of a given set of 
elements based upon position in the periodic table. 

Predict whether the bonding in a given substance is primarily covalent, metallic, or ionic, using the 
periodic table and the arrangement and energies of the elements' outermost electrons. 

CH004 

Know the basic constituents of matter. 

CH005 

Predict the shape of molecules (e.g., VSEPR theory, molecular geometry). 

CH006 

Know the basic subatomic constituents of matter (i.e., proton, neutron, electron). 

CH007 

Describe the Arrhenius, Bronsted-Lowry, and Lewis acid-base definitions. 

CH008 

Apply the pH scale to characterize aqueous acid and base solutions. 

CH009 

Classify and predict products for oxidation-reduction, acid-base and single and double replacement 
reactions. 

CH010 

CH011 

CH012 

Understand/know oxidation and reducing agents in an oxidation-reduction reaction. 

Measure and calculate the solute concentration of a solution in units (e.g., molarity, mass and 
volume percentage, mole fraction). 

Use stoichiometry to measure or calculate the relationship among mass, moles, and number of 
particles of reactants and products in a balanced chemical equation. 

CH013 

Experimentally measure and calculate limiting reactant and percent yield. 

CH014 

Understand/know why the chemical equilibrium of a reaction is a dynamic process. 

CH015 

Links chemical concepts to personal observations and experiences and chemically related news 
items. 


Physics 


ID 

Outcome 

PH001 

Describe, using words, graphs, or equations, the linear position, velocity, and acceleration of a 
given object for motion in one or two dimensions. 

PH002 

Describe, using words and a free-body diagram, the forces acting on a given object or a system of 
interacting objects. 

PH003 

Describe, using words, graphs, or equations the motion of a given object or system of interacting 
objects that results from forces acting upon the object or objects. 


Describe, using words, diagrams or equations, the linear momentum of a system under different 

PH004 

conditions, including before and after an interaction, using the principle of the conservation of 
linear momentum. 

PH005 

Describe, using words, graphs or equations, the motion of objects in a given system of interacting 
objects, using the principle of conservation of energy. 

PH006 

Describe, using words, graphs or equations, the change of energy within a given system of 
interacting objects, including the role of work and other forms of energy transfer. 


Describe, using words, graphs or equations, the motion of objects, before and after an interaction. 

PH007 

in a given system of interacting objects, using the principles of conservation of energy and 
momentum. 

PH008 

Describe, using words, graphs or equations, the change of linear momentum of a given system 
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under different conditions, including before and after an interaction, and any transfer of linear 
momentum into or out of the system, using the principle of the conservation of linear momentum. 
Describe, using words and diagrams, wavelength, amplitude, period, and frequency of a given 
wave. 


ID 

Outcome 

PH010 

Describe, using words, diagrams or equations, constructive and destructive interference using the 
principle of superposition. 

PH011 

Describe, using words, diagrams and graphs, amplitude, frequency, and energy of an object 
undergoing simple harmonic motion. 

PH012 

Describe, using words, diagrams, or equations, the potential and kinetic energy of a given 
oscillating system as a function of time. 

PH013 

Describe, using words, diagrams and equations, the forces on and motion of charges in given 
electric and magnetic fields. 

PH014 

Describe, using words and equations, the functional dependence of one variable on another by 
examining a graph. 

PH015 

Evaluate, using words or equations, reasonableness of calculated or measured physical 
parameters. 


Mathematics 


ID 

Outcome 

MA001 

Perform operations on matrices and use them to solve systems of linear equations. 

MA002 

Use techniques of Linear Programming for optimization problems. 

MA003 

Know how to find the characteristic values and vectors associated with a matrix. 

MA004 

Understand linear dependence and linear independence of a set of vectors. 

MA005 

Perform and interpret dot or scalar product and cross or vector product. 

MA006 

Have knowledge of gradient, curl, Jacobian, and Laplacian. 

MA007 

Can manipulate complex numbers in polar notation. 

MA008 

Use differentiation to optimize functions of one variable. 

MA009 

Understand implicit differentiation and its use for related rates problems. 

MA010 

Use differentials for approximation problems. 

MA011 

Can use techniques of integration to find arc lengths, areas and volumes. 

MA012 

Can differentiate and integrate in polar coordinates. 

MA013 

Evaluate line integrals and surface integrals. 

MA014 

Determine the convergence or divergence of improper integrals. 

MA015 

Use L’Hospital’s rules for determining certain limits. 

MA016 

Use Newtonians method for approximation. 

MA017 

Can expand functions in power series and know some important power series. 

MA018 

Use partial derivatives to optimize functions of two or more variables. 

MA019 

Use partial fractions and trigonometric substitutions for integration. 
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Computer Science 


ID Outcome 


CS001 

Demonstrate knowledge of and skill regarding the syntax and semantics of a high level 
programming language, its control structures, and its basic data representations 

CS002 

Demonstrate knowledge of and skill regarding common data abstraction mechanisms (e.g., data 
types or classes such as stacks, trees, etc.) 

CS003 

Know how to properly use classes, instances, methods, and attributes in programs. 

CS004 

Demonstrate knowledge of and skill regarding program correctness issues and practices (e.g., 
testing program results, test data design, loop invariants) 

CS005 

Design, implement, and test programs in languages from two different programming paradigms in 
a manner appropriate to each paradigm 

CS006 

Effectively use a variety of computing environments (e.g., single- and multi-user systems and 
various operating systems) 

CS007 

Describe the operation of a computer system-CPU & instruction cycle, peripherals, operating 
system, network components, and applications-indicating their purposes and interactions among 
them 

CS008 

Describe how data is represented at the machine level (e.g., character. Boolean, integer, floating 
point) 

CS009 

Identify and provide usage examples of the various data structures and files provided by a 
programming language (e.g., objects, various collections, files) 

CS010 

Describe the elements (people, hardware, software, etc.) And their interactions within information 
systems (database systems, the Web, etc.) 

CS011 

Demonstrate awareness cf social issues related to the use of computers in society and principles 
for making informed decisions regarding them (e.g., security, privacy, intellectual property, limits of 
computing, rapid change) 

CS012 

Be familiar with object oriented design principles and properly use them to develop programmatic 
solutions. 

CS013 

Recognize and apply computer programming related terminology. 

CS014 

Demonstrate clear understanding of programming language compilers and the software 
development process. 

CS015 

Implement syntactically correct, logically accurate programs based on logical algorithms/pseudo 
code. 

CS016 

Construct and implement commonly used algorithms and data structures for solving programming 
problems. 

CS017 

Create easily readable and understandable programs by employing appropriate modules and 
comments. 

CS018 

CS019 

CS020 

Design and implement computer-based solutions to problems in a variety of application areas.* 
Read and understand a large program consisting of several classes and interacting objects.* 
Design an interface.* 


Biology 


ID Outcome 


ID 

Outcome 

BI001 

Know the integration of individual structures and biochemical pathways that produce an organism 
capable of surviving and reproducing in its environment. 

BI002 

BI003 

BI004 

BI005 

Know the relationship between structure and function from the molecular to the organismal level. 
Know the structure and function of the components of a living system. 

Explain characteristics of living organisms. 

Explain the differences in cell structure and function between prokaryotes and eukaryotes. 
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ID Outcome 


BI006 

BI007 

BI008 

Explain how proteins are synthesized in a cell, using the processes of transcription and translation. 
Explain the relationship between feedback mechanisms and the maintenance of homeostasis. 
Explain how materials are moved within cells and across cell membranes. 

BI009 

Know/Understand the basic chemical principles including the properties of chemical bonds, 
oxidation and reduction, and the structure and function of the four major classes of biological 
macromolecules: carbohydrates, lipids, proteins, and nucleic acids. 

BI010 

Understand/Know the chemical and structural properties of DNA and its role in specifying the 
characteristics of an organism. 

BI011 

BI012 

Explain the relationships among DNA, chromosomes, genes, and alleles. 
Explain how traits are inherited and expressed in organisms. 

BI013 

Explain why mutations and new gene combinations may have positive, negative, or no effects on 
the organism. 

BI014 

Explain the role of mutation, genetic diversity, and natural selection in the adaptive potential of 
populations. 


Key Cognitive Strategies 


ID Outcome 


ID 

Outcome 

KCS01 

Know why curiosity, honesty, openness, and skepticism are so highly regarded in scientific 
research and how they are incorporated into the way engineers carry out science.* 

KCS02 

The student regards science as an essential way of knowing and learning engineering professions. 

KCS03 

Insist that the critical assumptions behind any line of reasoning be made explicit so that the validity 
of the position being taken-whether one's own or that of others-can be judged.* 

KCS04 

Know that there may be ethical Issues, such as ignoring contradicting data and selecting only 
supporting data, when considering claims. 

KCS05 

Know what is expected of a professional, such as knowing a professional code of ethics. 

KCS06 

Sample a wide range of scientific literature at a superficial level so as to build up an approximate 
framework for Interpreting what Is found on more detailed reading. 

KCS07 

Show an awareness that from time to time major shifts occur in the scientific view of how the world 
works, and that such a change is a strength of science and engineering, not a weakness. 

KCS08 

Suggest alternative ways of explaining data and criticize arguments in which data, explanations, or 
conclusions are represented as the only ones worth consideration, with no mention of other 
possibilities. 

KCS09 

Understand that scientists use methods and knowledge drawn from more than one scientific 
discipline as they solve problems. * 

KCS10 

Understand the relationships and common themes that connect mathematics, science, technology, 
and engineering. 

KCS11 

Apply knowledge and thinking skills to address real life problems and make informed decisions. 

KCS12 

Demonstrate problem-solving skills by developing a process for a solution using relationships 
between words, diagrams, graphs, and mathematical representations.* 

KCS13 

The student can plan a solution for a given problem by determining the steps necessary to solve, 
verify the solution, and evaluate whether the proposed solution Is reasonable. * 

KCS14 

The student can determine known and unknown quantities that are relevant to a given problem. * 

KCS15 

Design a testable scientific hypothesis, refine the hypothesis, and conduct an experiment to 
confirm or refute the hypothesis. 

KCS16 

Demonstrate the use of experimental thinking as an inherent part of the problem solving process. 

KCS17 

Persist in quest for solutions by designing alternative solutions to problems and accepting failure 
and ambiguity as part of the process. 

KCS18 

KCS19 

Make up and write out simple algorithms for solving problems that take multiple steps. 
Understand attributes, purposes, and limitations of scientific models. * 
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KCS20 

KCS21 

KCS22 

KCS23 


Understand that precision in measurement, description, and reporting is an important component 
of scientific research.* 

Be able to estimate how likely it is that some event of interest might have occurred just by chance. 
Determine whether the mathematical solutions are reasonable by reviewing the process and 
checking against typical values from experience of the everyday world. 

Trace the source of any large disparity between an estimate and the calculated answer. 


ID 

Outcome 

KCS24 

Offers clear, thorough explanations by beginning with simple generalizations and moving toward 
complexity and specificity. 

KCS25 

KCS26 

Use tables, charts, and graphs in making arguments and claims in oral and written presentations.* 
Check graphs to see that they do not misrepresent results by using inappropriate scales or by 
failing to specify the axes clearly. 

Describe, in words, both written and orally, an experiment or derivation he or she has made to an 

KCS27 

audience of non-experts in a manner that avoids jargon and follows a clear logical path between 
question, method, solution, and impact.* 

KCS28 

Participate in group discussions on scientific topics by restating or summarizing accurately what 
others have said, asking for clarification or elaboration, and expressing alternative positions.* 

KCS29 

Use and correctly interpret relational terms such as if . . . then . . ., and, or, sufficient, necessary, 
some, every, not, correlates with, and causes.* 

KCS30 

Write scientific analyses with clarity, cohesiveness, and meaning, especially as they present and 
defend experiments. 

KCS31 

Write clear, step-by-step instructions for conducting investigations, operating something, or 
following a procedure. 


Teaching Practices 


ID 

Outcome 

TP001 

Providing timely and suitable feedback to students. 

TP002 

Employing announced grading criteria that are consistent with course goals. 

TP003 

Demonstrating high but appropriate standards in grading. 

TP004 

Helping and encouraging students to develop their abilities. 

TP005 

Working to develop students' good study habits, self-reliance and responsibility for learning. 

TP006 

Ensuring that all students have equal access and opportunity to learn and apply science, 
technology, and mathematics to engineering. 

TP007 

Demonstrating an attitude that all students will meet high expectations. 

TP008 

TP009 

Developing goals and pace for the course that are appropriately challenging. 

Connecting academic ideas with the way a professional in the field thinks and explains the 
practical issues of the topics covered. 

TP010 

Facilitating student engagement in active exploration of the problem, finding materials, and 
proposing, testing and modifying solutions. 

TP011 

Including assignments that promote deep understanding of fundamental thought processes. 

TP012 

Raising questions consciously that cause students to consider the quality of evidence. 
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Appendix B: Ratings Results - General 

Engineering Courses 

Question 1 - What do students need to know to enter my class? (115 courses 
participating) 

Scale: 

1 - No prior student knowledge of content necessary 

2 - Basic understanding of content necessary 

3 - Thorough understanding of content necessary 

4 - Mastery of content necessary 

N/A - This expectation is not a part of this course 

Question 2 - What I observe students know (114 courses participating) 

Scale: 

1 - Student knowledge is adequate relative to necessary levels 

2 - Student knowledge is less than adequate relative to necessary levels 

Question 3 - How important is this for students to successfully complete this course? (118 
courses participating) 

Scale: 

1 - Least Important 

2 - Less Important 

3 - More Important 

4 - Most Important 

Question 4 - How important is the PE in preparing students for further engineering 
studies? (57 courses participating) 

Scale: 

1 - Least Important 

2 - Less Important 

3 - More Important 

4 - Most Important 
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General Engineering Outcomes 


Question 1 Question 2 Question 3 Question 4 

Mean St. % % Adequate Mean St. Mean St. 

Score Dev N/A Knowledge Score Dev Score Dev 


GE001 

2.39 

0.99 

17% 

73% 

3.05 

0.72 

3.23 

0.79 

GE002 

2.13 

1.02 

26% 

59% 

2.78 

0.77 

2.86 

0.84 

GE003 

1.95 

0.96 

30% 

60% 

2.68 

0.74 

2.61 

0.90 

GE004 

1.94 

0.93 

19% 

73% 

2.89 

0.83 

3.04 

0.87 

GE005 

1.85 

0.97 

25% 

70% 

3.08 

0.81 

3.24 

0.82 

GE006 

1.75 

0.86 

33% 

61% 

2.99 

0.88 

3.02 

0.86 

GE007 

1.77 

0.89 

46% 

65% 

2.97 

0.79 

2.77 

0.94 

GE008 

1.77 

1.00 

20% 

62% 

2.93 

0.85 

3.09 

0.81 

GE009 

2.11 

0.94 

35% 

59% 

3.04 

0.81 

3.09 

0.81 

GE010 

1.97 

0.93 

43% 

77% 

3.13 

0.88 

2.73 

1.03 

GE011 

2.15 

1.00 

41% 

62% 

3.03 

0.86 

2.86 

1.00 

GE012 

2.00 

0.97 

50% 

53% 

3.00 

0.78 

3.10 

0.88 

GE013 

2.04 

0.82 

39% 

51% 

3.14 

0.70 

3.07 

0.75 

GE014 

1.98 

0.85 

57% 

51% 

3.00 

0.87 

2.85 

0.85 

GE015 

1.76 

0.84 

48% 

61% 

2.78 

0.88 

2.63 

0.82 

GE016 

2.57 

1.06 

26% 

59% 

3.07 

0.82 

3.35 

0.62 

GE017 

2.70 

1.10 

35% 

71% 

3.10 

0.73 

3.08 

0.76 

GE018 

2.45 

1.06 

22% 

63% 

3.24 

0.78 

3.50 

0.67 

GE019 

2.03 

0.93 

43% 

56% 

2.97 

0.76 

2.84 

0.83 

GE020 

2.29 

1.07 

33% 

59% 

3.08 

0.81 

3.16 

0.73 

GE021 

2.69 

1.17 

34% 

64% 

3.22 

0.80 

3.13 

0.97 

GE022 

2.24 

1.02 

39% 

64% 

3.27 

0.73 

3.20 

0.81 

GE023 

2.49 

1.09 

16% 

62% 

3.29 

0.81 

3.37 

0.75 

GE024 

2.53 

1.13 

28% 

54% 

3.13 

0.86 

3.10 

0.80 

GE025 

2.47 

1.06 

57% 

43% 

3.13 

0.72 

2.90 

0.84 

Total 

2.17 

1.03 


62% 

3.04 

0.81 

3.03 

0.86 
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Chemistry Outcomes 



Mean 

Score 

Question 1 

St. 

Dev 

% 

N/A 

Question 2 

% Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

CH001 

1.89 

0.78 

17% 

44% 

2.89 

0.93 

2.54 

0.88 

CH002 

1.88 

1.13 

18% 

50% 

3.13 

0.99 

2.54 

1.05 

CH003 

2.00 

1.15 

19% 

57% 

3.14 

1.07 

2.46 

1.13 

CH004 

2.00 

1.00 

16% 

55% 

3.27 

0.79 

2.77 

1.01 

CH005 

1.60 

0.89 

21% 

60% 

3.00 

1.00 

2.23 

1.09 

CH006 

2.40 

1.34 

21% 

40% 

3.20 

1.10 

2.77 

1.09 

CH007 

1.50 

0.58 

21% 

50% 

3.25 

0.96 

2.23 

1.09 

CH008 

1.83 

0.75 

20% 

67% 

2.83 

0.98 

2.54 

1.05 

CH009 

1.40 

0.89 

21% 

80% 

3.20 

0.84 

2.46 

1.13 

CH010 

1.60 

0.89 

21% 

40% 

3.00 

1.00 

2.46 

0.97 

CH011 

2.17 

1.17 

20% 

67% 

3.33 

0.82 

2.54 

1.13 

CH012 

2.33 

1.21 

20% 

60% 

3.40 

0.89 

2.77 

1.17 

CH013 

1.80 

1.10 

21% 

40% 

3.40 

0.89 

2.54 

1.13 

CH014 

1.83 

0.75 

20% 

50% 

3.33 

0.82 

2.69 

1.25 

CH015 

2.00 

0.71 

21% 

60% 

3.20 

0.84 

2.54 

1.05 

Total 

1.90 

0.95 


50% 

3.16 

0.87 

2.54 

1.06 
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Physics Outcomes 



Mean 

Score 

Question 1 

St. 

Dev 

% 

N/A 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

PH001 

2.14 

1.03 

23% 

64% 

3.00 

0.78 

3.00 

0.89 

PH002 

2.40 

1.24 

22% 

47% 

3.60 

0.51 

3.29 

0.90 

PH003 

1.83 

1.11 

24% 

50% 

3.09 

0.83 

2.90 

1.07 

PH004 

2.00 

1.07 

28% 

63% 

3.71 

0.49 

2.95 

1.05 

PH005 

2.00 

1.00 

26% 

56% 

3.56 

0.73 

3.10 

0.97 

PH006 

2.21 

1.05 

22% 

43% 

3.00 

0.88 

3.05 

0.89 

PH007 

1.64 

0.67 

24% 

55% 

3.00 

0.94 

2.95 

1.05 

PH008 

1.63 

0.74 

27% 

63% 

3.25 

0.89 

2.95 

1.15 

PH009 

2.57 

1.40 

27% 

71% 

2.86 

0.69 

2.61 

1.14 

PH010 

2.63 

1.30 

26% 

50% 

2.71 

0.76 

2.42 

1.12 

PH011 

1.88 

1.36 

26% 

50% 

2.13 

0.64 

2.39 

1.04 

PH012 

2.29 

1.38 

28% 

71% 

2.83 

0.98 

2.74 

1.05 

PH013 

2.33 

1.37 

29% 

50% 

2.20 

0.84 

2.61 

0.98 

PH014 

2.63 

1.15 

20% 

67% 

3.27 

0.59 

3.05 

1.00 

PH015 

2.05 

1.13 

17% 

35% 

3.28 

0.89 

3.38 

0.86 

Total 

2.15 

1.12 


54% 

3.10 

0.83 

2.91 

1.03 
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Mathematics Outcomes 



Mean 

Score 

Question 1 

St. 

Dev 

% 

N/A 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

MA001 

2.60 

1.04 

22% 

61% 

3.14 

0.76 

2.82 

0.97 

MA002 

2.25 

1.22 

37% 

55% 

3.27 

0.65 

2.50 

0.97 

MA003 

2.56 

1.31 

33% 

27% 

3.00 

0.85 

2.59 

1.01 

MA004 

2.62 

1.26 

35% 

33% 

3.08 

0.90 

2.55 

0.99 

MA005 

2.85 

1.14 

34% 

69% 

3.21 

0.70 

2.55 

0.96 

MA006 

2.00 

1.00 

44% 

33% 

3.33 

0.58 

2.10 

0.94 

MA007 

2.85 

1.28 

35% 

67% 

3.17 

0.83 

2.35 

1.11 

MA008 

2.79 

0.92 

30% 

56% 

2.95 

0.78 

2.84 

1.05 

MA009 

2.91 

0.54 

37% 

55% 

3.18 

0.60 

2.60 

1.07 

MA010 

2.75 

0.46 

39% 

38% 

3.13 

0.83 

2.36 

1.13 

MA011 

2.53 

0.72 

31% 

65% 

2.82 

0.81 

2.73 

1.01 

MA012 

2.20 

1.03 

38% 

33% 

3.11 

0.93 

2.21 

1.05 

MA013 

2.50 

1.08 

37% 

50% 

3.20 

0.79 

2.37 

1.03 

MA014 

1.40 

0.55 

43% 

60% 

2.60 

0.55 

2.00 

0.92 

MA015 

2.40 

0.89 

43% 

80% 

3.20 

0.45 

1.93 

0.92 

MA016 

2.00 

1.13 

35% 

31% 

2.46 

0.78 

2.33 

0.88 

MA017 

2.50 

1.35 

38% 

30% 

2.50 

0.85 

2.22 

0.97 

MA018 

2.40 

1.17 

38% 

30% 

3.00 

0.82 

2.43 

1.01 

MA019 

2.12 

1.05 

31% 

29% 

2.88 

0.70 

2.48 

1.03 

Total 

2.50 

1.07 


48% 

3.00 

0.78 

2.43 

1.02 
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Computer Science Outcomes 



Mean 

Score 

Question 1 

St. 

Dev 

% 

N/A 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

CS001 

1.78 

0.67 

23% 

38% 

2.56 

0.73 

2.11 

0.90 

CS002 

1.50 

0.84 

26% 

33% 

2.33 

0.52 

1.89 

0.90 

CS003 

1.00 

0.00 

29% 

0% 

2.00 

0.00 

1.67 

0.91 

CS004 

1.78 

1.09 

23% 

56% 

2.44 

0.73 

1.89 

0.90 

CS005 

1.50 

0.58 

28% 

50% 

2.25 

0.50 

1.72 

0.83 

CS006 

1.43 

0.53 

25% 

43% 

2.29 

0.95 

2.06 

0.87 

CS007 

1.63 

1.19 

24% 

14% 

2.63 

0.92 

2.17 

1.20 

CS008 

1.57 

1.13 

25% 

29% 

2.43 

0.98 

1.94 

1.00 

CS009 

1.33 

0.82 

24% 

50% 

2.50 

1.05 

1.88 

0.99 

CS010 

1.75 

0.89 

23% 

25% 

2.63 

0.92 

2.17 

1.04 

CS011 

1.75 

1.22 

21% 

42% 

2.50 

0.80 

2.17 

0.79 

CS012 

1.14 

0.38 

25% 

71% 

2.43 

0.98 

2.00 

0.91 

CS013 

1.73 

1.10 

22% 

55% 

2.45 

0.82 

2.44 

0.92 

CS014 

1.57 

0.79 

25% 

29% 

2.29 

0.49 

2.11 

1.02 

CS015 

1.63 

0.74 

24% 

25% 

2.63 

0.52 

1.83 

0.99 

CS016 

1.67 

0.82 

26% 

17% 

2.60 

0.89 

2.06 

1.00 

CS017 

1.63 

0.92 

23% 

38% 

2.63 

0.74 

2.22 

0.94 

CS018 

1.25 

0.46 

24% 

43% 

2.25 

0.46 

2.33 

0.91 

CS019 

1.25 

0.50 

28% 

75% 

2.00 

0.00 

1.72 

0.83 

CS020 

1.33 

0.52 

26% 

40% 

2.33 

1.03 

1.61 

0.92 

Total 

1.56 

0.84 


39% 

2.44 

0.75 

2.00 

0.94 
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Key Cognitive Strategies 



Mean 

Score 

Question 1 

St. 

Dev 

% 

N/A 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

KCS01 

2.22 

0.94 

25% 

67% 

2.90 

0.67 

2.85 

0.74 

KCS02 

2.26 

0.92 

23% 

71% 

2.87 

0.81 

2.98 

0.75 

KCS03 

2.03 

0.90 

30% 

50% 

2.79 

0.91 

2.92 

0.83 

KCS04 

1.77 

0.80 

33% 

69% 

2.88 

0.98 

3.06 

0.76 

KCS05 

1.93 

1.01 

28% 

75% 

3.00 

0.93 

3.24 

0.74 

KCS06 

2.05 

0.97 

58% 

46% 

2.89 

0.84 

2.80 

0.83 

KCS07 

2.07 

1.03 

51% 

65% 

2.98 

0.89 

2.80 

0.93 

KCS08 

1.86 

0.83 

46% 

55% 

2.96 

0.83 

2.89 

0.88 

KCS09 

1.97 

0.89 

33% 

69% 

2.83 

0.89 

2.94 

0.81 

KCS10 

1.99 

1.01 

23% 

66% 

2.91 

0.88 

3.06 

0.67 

KCS11 

2.02 

0.93 

15% 

74% 

3.17 

0.67 

3.06 

0.72 

KCS12 

2.02 

1.01 

15% 

69% 

3.15 

0.91 

3.41 

0.70 

KCS13 

1.99 

0.99 

13% 

69% 

3.02 

0.90 

3.34 

0.67 

KCS14 

2.19 

1.00 

23% 

68% 

3.14 

0.93 

3.28 

0.77 

KCS15 

1.94 

0.91 

59% 

52% 

3.21 

0.74 

2.89 

0.83 

KCS16 

1.95 

0.98 

37% 

63% 

2.88 

0.92 

2.89 

0.85 

KCS17 

1.80 

0.92 

38% 

63% 

2.81 

0.91 

2.96 

0.87 

KCS18 

2.22 

0.90 

45% 

56% 

3.22 

0.69 

2.87 

0.81 

KCS19 

2.08 

0.98 

45% 

52% 

3.08 

0.74 

2.8 

0.79 

KCS20 

2.07 

0.99 

26% 

68% 

3.00 

0.76 

3.11 

0.77 

KCS21 

2.06 

0.96 

63% 

62% 

2.94 

0.77 

2.42 

0.81 

KCS22 

2.09 

1.03 

31% 

69% 

3.09 

0.74 

3.15 

0.75 

KCS23 

2.03 

0.99 

30% 

68% 

3.03 

0.80 

3.04 

0.74 

KCS24 

2.07 

0.96 

41% 

63% 

3.02 

0.72 

2.89 

0.75 

KCS25 

2.26 

1.11 

37% 

65% 

3.24 

0.78 

3.18 

0.81 

KCS26 

2.19 

1.04 

36% 

57% 

3.23 

0.76 

3.08 

0.79 

KCS27 

2.23 

1.07 

55% 

61% 

3.08 

0.75 

2.79 

0.81 

KCS28 

2.05 

1.02 

54% 

62% 

3.03 

0.80 

2.76 

0.88 

KCS29 

2.29 

0.99 

58% 

66% 

3.11 

0.83 

2.69 

0.97 

KCS30 

1.97 

0.91 

57% 

59% 

3.08 

0.87 

2.78 

0.99 

KCS31 

1.93 

0.94 

42% 

63% 

2.96 

0.95 

3.15 

0.84 

Total 

2.05 

0.97 


65% 

3.01 

0.84 

2.98 

0.82 
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Teaching Practices 



Question 1 

Mean St. 

Score Dev 

% 

N/A 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

TP001 



3.67 

0.49 

3.54 

0.54 

TP002 



3.58 

0.66 

3.42 

0.71 

TP003 



3.53 

0.65 

3.35 

0.60 

TP004 



3.74 

0.52 

3.72 

0.50 

TP005 



3.54 

0.60 

3.52 

0.55 

TP006 



3.43 

0.74 

3.36 

0.82 

TP007 


N/A 

3.62 

0.59 

3.46 

0.62 

TP008 



3.56 

0.57 

3.44 

0.62 

TP009 



3.64 

0.56 

3.43 

0.62 

TP010 



3.42 

0.71 

3.19 

0.68 

TP011 



3.44 

0.73 

3.23 

0.73 

TP012 



3.23 

0.81 

3.15 

0.79 

Total 



3.53 

0.65 

3.4 

0.67 
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Appendix C: Ratings Results - 

Chemistry Courses 

These questions use the same ratings scales as presented in Appendix B. 

Chemistry Outcomes 



Question 1 
Mean 

Score St. Dev 

Question 2 

% Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

CH001 

2.47 

1.37 

53% 

3.72 

0.46 

3.40 

0.70 

CH002 

2.47 

1.33 

53% 

3.72 

0.46 

3.44 

0.53 

CH003 

2.53 

1.23 

65% 

3.67 

0.59 

3.33 

0.71 

CH004 

2.82 

1.33 

65% 

3.67 

0.49 

3.44 

0.53 

CH005 

2.00 

1.31 

47% 

3.50 

0.63 

3.11 

0.78 

CH006 

2.87 

1.46 

67% 

3.81 

0.40 

3.22 

0.83 

CH007 

2.46 

1.39 

58% 

3.57 

0.51 

2.89 

1.05 

CH008 

2.31 

1.38 

46% 

3.50 

0.52 

3.00 

1.00 

CH009 

2.40 

1.30 

47% 

3.38 

0.50 

3.22 

0.67 

CH010 

2.40 

1.30 

40% 

3.56 

0.51 

3.00 

1.00 

CH011 

2.20 

1.26 

47% 

3.38 

0.62 

3.44 

0.53 

CH012 

2.43 

1.28 

43% 

3.60 

0.51 

3.50 

0.53 

CHOI 3 

2.54 

1.33 

58% 

3.36 

0.63 

3.50 

0.53 

CH014 

2.33 

1.37 

42% 

3.54 

0.52 

3.13 

0.99 

CH015 

2.63 

1.20 

63% 

3.65 

0.49 

3.50 

0.53 

Total 

2.46 

1.30 


3.58 

0.53 

3.27 

0.74 
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Key Cognitive Strategies 



Question 1 

Mean 

Score St. Dev 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

KCS01 

2.64 

0.92 

45% 

3.55 

0.52 

3.83 

0.41 

KCS02 

3.00 

1.07 

75% 

3.38 

0.52 

3.57 

0.79 

KCS03 

2.17 

1.27 

50% 

3.50 

0.52 

3.43 

0.79 

KCS04 

2.31 

1.11 

69% 

3.54 

0.88 

3.86 

0.38 

KCS05 

2.18 

1.33 

73% 

3.40 

0.97 

3.50 

0.84 

KCS06 

2.44 

1.13 

56% 

3.33 

1.00 

3.17 

1.33 

KCS07 

2.38 

1.04 

69% 

3.46 

0.88 

3.50 

0.84 

KCS08 

2.33 

0.98 

42% 

3.42 

0.51 

3.43 

1.13 

KCS09 

2.62 

1.04 

69% 

3.23 

0.83 

3.43 

1.13 

KCS10 

2.69 

1.11 

67% 

3.31 

0.85 

3.43 

1.13 

KCS11 

2.54 

1.13 

54% 

3.38 

0.87 

3.43 

1.13 

KCS12 

2.58 

1.16 

50% 

3.75 

0.45 

3.57 

1.13 

KCS13 

2.58 

1.00 

50% 

3.75 

0.45 

3.86 

0.38 

KCS14 

2.58 

1.00 

50% 

3.75 

0.45 

3.57 

1.13 

KCS15 

2.33 

1.15 

50% 

3.42 

0.67 

3.29 

1.25 

KCS16 

2.46 

1.20 

23% 

3.54 

0.66 

3.71 

0.76 

KCS17 

2.25 

1.22 

42% 

3.25 

0.87 

3.14 

1.46 

KCS18 

2.58 

1.16 

50% 

3.67 

0.49 

3.29 

1.11 

KCS19 

2.46 

1.20 

54% 

3.46 

0.88 

3.43 

1.13 

KCS20 

2.54 

1.13 

50% 

3.62 

0.65 

3.71 

0.76 

KCS21 

2.60 

1.07 

60% 

3.50 

0.53 

3.33 

0.82 

KCS22 

2.45 

1.04 

45% 

3.80 

0.42 

3.57 

0.79 

KCS23 

2.36 

1.12 

36% 

3.45 

0.82 

3.57 

0.79 

KCS24 

2.40 

0.97 

20% 

3.50 

0.53 

3.14 

1.07 

KCS25 

2.45 

1.13 

36% 

3.55 

0.52 

3.29 

1.11 

KCS26 

1.89 

0.93 

0% 

3.78 

0.44 

3.43 

1.13 

KCS27 

2.90 

0.99 

50% 

3.80 

0.42 

3.43 

1.13 

KCS28 

2.73 

1.10 

73% 

3.82 

0.40 

3.29 

1.11 

KCS29 

2.70 

1.16 

40% 

3.70 

0.48 

3.17 

1.17 

KCS30 

2.46 

1.20 

8% 

3.77 

0.60 

3.43 

0.79 

KCS31 

2.73 

1.10 

36% 

3.82 

0.40 

3.43 

1.13 

Total 

2.49 

1.08 


3.55 

0.66 

3.46 

0.95 
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Teaching Practices 



Question 1 Question 2 

% with 

Mean Adequate 

Score St. Dev Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

TP001 


4.00 

0.00 

4.00 

0.00 

TP002 


4.00 

0.00 

4.00 

0.00 

TP003 


4.00 

0.00 

4.00 

0.00 

TP004 


4.00 

0.00 

4.00 

0.00 

TP005 


3.83 

0.39 

4.00 

0.00 

TP006 


3.83 

0.39 

3.80 

0.45 

TP007 

N/A 

3.00 

0.95 

3.60 

0.89 

TP008 


4.00 

0.00 

4.00 

0.00 

TP009 


3.58 

0.67 

3.80 

0.45 

TP010 


3.67 

0.49 

3.80 

0.45 

TP011 


3.75 

0.45 

3.80 

0.45 

TP012 


3.83 

0.39 

4.00 

0.00 

Total 


3.79 

0.50 

3.90 

0.35 
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Appendix D: Ratings Results - Physics 

Courses 


These questions use the same ratings scales as presented in Appendix B. 

Physics Outcomes 



Question 1 
Mean 

Score St. Dev 

Question 2 

% Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

PH001 

2.21 

0.98 

78% 

3.39 

0.83 

3.33 

0.59 

PH002 

1.90 

1.08 

70% 

3.50 

0.88 

3.53 

0.62 

PH003 

2.14 

1.09 

70% 

3.61 

0.79 

3.53 

0.62 

PH004 

1.79 

1.05 

70% 

3.33 

0.92 

3.38 

0.81 

PH005 

1.75 

0.97 

69% 

3.54 

0.65 

3.47 

0.80 

PH006 

1.68 

0.98 

69% 

3.63 

0.63 

3.31 

0.87 

PH007 

1.68 

0.98 

69% 

3.48 

0.58 

3.38 

0.81 

PH008 

1.79 

0.99 

69% 

3.33 

0.92 

3.53 

0.52 

PH009 

1.55 

0.96 

76% 

3.64 

0.49 

3.33 

0.62 

PH010 

1.48 

0.75 

65% 

3.43 

0.60 

3.13 

0.64 

PH011 

1.38 

0.74 

65% 

3.67 

0.48 

3.31 

0.60 

PH012 

1.48 

0.73 

71% 

3.45 

0.74 

3.19 

0.66 

PH013 

1.41 

0.80 

62% 

3.77 

0.43 

3.25 

0.68 

PH014 

2.35 

0.94 

76% 

3.35 

0.75 

3.31 

0.79 

PH015 

1.96 

0.77 

58% 

3.35 

0.63 

3.38 

0.89 

Total 

1.79 

0.97 


3.49 

0.72 

3.36 

0.70 
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Key Cognitive Strategies 



Question 1 

Mean 

Score St. Dev 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

KCS01 

2.11 

0.96 

67% 

2.71 

0.92 

2.92 

0.95 

KCS02 

2.15 

0.99 

76% 

2.89 

1.05 

3.15 

1.07 

KCS03 

1.91 

0.73 

70% 

3.14 

0.71 

3.54 

0.52 

KCS04 

2.00 

0.84 

69% 

2.82 

0.73 

3.08 

0.86 

KCS05 

1.88 

1.25 

67% 

2.29 

0.95 

3.00 

0.82 

KCS06 

2.20 

0.45 

20% 

2.50 

0.58 

2.45 

1.13 

KCS07 

1.89 

0.74 

59% 

2.65 

0.79 

2.55 

0.82 

KCS08 

1.63 

0.62 

57% 

2.57 

0.65 

2.75 

0.87 

KCS09 

2.18 

0.81 

80% 

2.27 

0.96 

2.42 

0.79 

KCS10 

1.89 

0.90 

60% 

2.59 

0.62 

2.75 

0.75 

KCS11 

1.84 

0.83 

69% 

2.67 

0.97 

3.42 

0.51 

KCS12 

1.74 

0.69 

65% 

3.82 

0.39 

3.45 

0.52 

KCS13 

1.68 

0.57 

63% 

3.48 

0.68 

3.33 

0.89 

KCS14 

1.96 

0.88 

70% 

3.45 

0.67 

3.33 

0.49 

KCS15 

1.71 

0.64 

61% 

2.90 

0.72 

2.92 

1.00 

KCS16 

1.69 

0.70 

85% 

2.80 

0.77 

2.67 

0.78 

KCS17 

1.53 

0.52 

36% 

2.71 

0.83 

2.58 

0.79 

KCS18 

1.55 

0.82 

40% 

3.10 

0.74 

2.83 

0.72 

KCS19 

1.73 

0.63 

63% 

3.05 

0.74 

3.08 

0.90 

KCS20 

1.68 

0.78 

74% 

2.62 

0.92 

2.92 

0.90 

KCS21 

1.73 

0.90 

56% 

2.30 

0.95 

2.50 

1.09 

KCS22 

1.68 

0.67 

69% 

3.06 

0.64 

3.08 

0.90 

KCS23 

1.72 

0.57 

63% 

2.94 

0.75 

3.25 

0.75 

KCS24 

1.71 

0.77 

62% 

2.73 

0.70 

2.91 

0.94 

KCS25 

1.80 

0.77 

69% 

3.00 

0.78 

3.17 

0.72 

KCS26 

1.89 

0.94 

75% 

2.83 

0.79 

3.17 

0.72 

KCS27 

1.33 

0.50 

29% 

2.63 

1.06 

2.55 

1.21 

KCS28 

1.60 

1.07 

25% 

3.22 

0.97 

2.45 

0.93 

KCS29 

2.36 

1.29 

67% 

2.70 

0.82 

2.92 

0.90 

KCS30 

1.56 

0.62 

50% 

2.65 

1.06 

3.08 

0.90 

KCS31 

1.79 

0.70 

54% 

2.71 

0.91 

2.92 

0.90 

Total 

1.81 

0.79 


2.89 

0.85 

2.95 

0.88 
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Teaching Practices 



Question 1 Question 2 

% with 

Mean Adequate 

Score St. Dev Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

TP001 


3.52 

0.59 

3.00 

1.11 

TP002 


3.42 

0.78 

3.38 

0.77 

TP003 


3.33 

1.01 

3.38 

0.87 

TP004 


3.58 

0.65 

3.77 

0.44 

TP005 


3.63 

0.49 

3.62 

0.51 

TP006 


3.36 

0.79 

3.42 

0.67 

TP007 

N/A 

3.22 

0.90 

3.38 

0.87 

TP008 


3.71 

0.46 

3.46 

0.52 

TP009 


2.64 

0.95 

3.23 

0.60 

TP010 


3.08 

0.88 

3.38 

0.65 

TP011 


3.42 

0.72 

3.46 

0.78 

TP012 


3.08 

0.72 

3.33 

0.65 

Total 


3.34 

0.80 

3.40 

0.73 
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Appendix E: Ratings Results - 
Mathematics Courses 


These questions use the same ratings scales as presented in Appendix B. 


Mathematics Outcomes 



Question 1 
Mean 

Score St. Dev 

Question 2 

% Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

MA001 

1.88 

0.74 

58% 

2.46 

0.72 

2.75 

0.74 

MA002 

2.08 

1.08 

50% 

2.50 

1.00 

2.43 

0.99 

MA003 

1.33 

0.65 

36% 

3.17 

1.03 

2.36 

0.95 

MA004 

1.86 

1.03 

69% 

2.93 

1.00 

2.35 

0.88 

MA005 

1.71 

0.96 

55% 

3.24 

0.83 

2.65 

0.93 

MA006 

1.50 

0.94 

62% 

3.50 

0.65 

2.39 

0.99 

MA007 

1.75 

0.86 

40% 

2.60 

0.74 

2.24 

0.77 

MA008 

2.61 

1.20 

79% 

3.03 

0.89 

2.85 

0.92 

MA009 

2.53 

1.08 

61% 

2.81 

0.81 

2.77 

0.91 

MA010 

1.95 

0.92 

50% 

2.58 

0.72 

2.52 

0.82 

MA011 

2.09 

1.03 

60% 

2.88 

0.83 

2.52 

0.96 

MA012 

1.65 

0.83 

61% 

3.00 

0.87 

2.40 

0.96 

MA013 

1.63 

0.81 

50% 

3.13 

0.72 

2.50 

1.02 

MA014 

1.88 

0.95 

65% 

3.00 

0.78 

2.48 

0.87 

MA015 

2.18 

1.06 

61% 

2.71 

0.74 

2.44 

0.80 

MA016 

2.07 

1.04 

50% 

2.38 

0.71 

2.38 

0.85 

MA017 

1.95 

1.00 

50% 

2.80 

0.89 

2.29 

0.91 

MA018 

1.94 

0.93 

56% 

3.19 

0.83 

2.63 

0.92 

MA019 

1.86 

0.89 

54% 

2.75 

0.89 

2.20 

0.96 

Total 

2.00 

1.01 


2.85 

0.85 

2.48 

0.91 
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Key Cognitive Strategies 



Question 1 

Mean 

Score St. Dev 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

KCS01 

2.10 

0.94 

52% 

2.90 

0.70 

3.17 

0.78 

KCS02 

2.26 

0.81 

41% 

2.72 

0.67 

2.96 

0.64 

KCS03 

2.03 

0.89 

58% 

3.03 

0.80 

3.27 

0.60 

KCS04 

1.91 

0.83 

55% 

2.91 

0.70 

3.14 

0.89 

KCS05 

2.25 

1.16 

50% 

2.88 

0.64 

2.86 

1.08 

KCS06 

1.89 

1.05 

44% 

2.67 

0.50 

2.59 

0.73 

KCS07 

1.64 

0.74 

57% 

2.50 

0.85 

2.61 

0.78 

KCS08 

2.41 

1.28 

35% 

2.59 

0.80 

2.73 

0.94 

KCS09 

2.22 

1.17 

59% 

2.45 

0.60 

2.91 

0.67 

KCS10 

2.00 

0.91 

59% 

2.56 

0.66 

2.96 

0.73 

KCS11 

2.24 

1.02 

43% 

3.08 

0.80 

3.32 

0.69 

KCS12 

2.50 

0.90 

65% 

3.36 

0.58 

3.48 

0.71 

KCS13 

2.20 

0.93 

60% 

3.45 

0.63 

3.62 

0.64 

KCS14 

2.59 

1.09 

56% 

3.23 

0.72 

3.48 

0.65 

KCS15 

2.25 

1.28 

57% 

2.71 

0.95 

3.09 

0.75 

KCS16 

2.55 

0.99 

57% 

2.57 

0.69 

2.91 

1.00 

KCS17 

2.19 

0.93 

35% 

2.81 

0.70 

3.17 

0.82 

KCS18 

2.61 

0.95 

60% 

3.10 

0.72 

2.88 

0.85 

KCS19 

2.29 

1.05 

65% 

2.75 

0.86 

3.13 

0.81 

KCS20 

2.39 

1.24 

76% 

2.59 

0.62 

3.04 

0.82 

KCS21 

1.88 

0.99 

88% 

2.88 

1.13 

2.62 

0.92 

KCS22 

2.82 

0.94 

68% 

3.34 

0.58 

3.56 

0.51 

KCS23 

2.49 

0.85 

44% 

2.94 

0.74 

3.04 

0.77 

KCS24 

2.14 

0.83 

50% 

2.95 

0.67 

2.83 

0.78 

KCS25 

2.41 

1.05 

56% 

2.94 

0.75 

3.36 

0.64 

KCS26 

2.61 

0.99 

61% 

3.04 

0.81 

3.21 

0.88 

KCS27 

2.20 

1.03 

80% 

3.60 

0.70 

3.14 

0.99 

KCS28 

1.92 

0.86 

46% 

3.08 

0.76 

3.14 

0.89 

KCS29 

2.36 

1.03 

50% 

3.09 

0.59 

3.08 

0.95 

KCS30 

2.22 

0.67 

33% 

2.44 

0.53 

2.95 

0.65 

KCS31 

3.05 

1.08 

67% 

3.06 

0.73 

2.91 

0.79 

Total 

2.34 

1.00 


2.97 

0.75 

3.08 

0.82 
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Teaching Practices 



Question 1 Question 2 

% with 

Mean Adequate 

Score St. Dev Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

TP001 


3.49 

0.51 

3.52 

0.79 

TP002 


3.43 

0.62 

3.35 

0.78 

TP003 


3.59 

0.54 

3.48 

0.59 

TP004 


3.72 

0.46 

3.48 

0.60 

TP005 


3.17 

0.80 

3.29 

0.85 

TP006 


3.21 

0.74 

3.37 

0.60 

TP007 

N/A 

3.34 

0.57 

3.50 

0.61 

TP008 


3.39 

0.58 

3.38 

0.50 

TP009 


2.77 

0.77 

2.90 

0.79 

TP010 


3.18 

0.75 

3.38 

0.74 

TP011 


3.48 

0.62 

3.29 

0.85 

TP012 


2.91 

0.89 

3.10 

0.70 

Total 


3.31 

0.71 

3.34 

0.71 
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Appendix F: Ratings Results - 
Computer Science Courses 


These questions use the same ratings scales as presented in Appendix B. 

Computer Science Outcomes 



Question 1 
Mean 

Score St. Dev 

Question 2 

% Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

CS001 

2.22 

1.24 

86% 

3.54 

0.72 

3.44 

0.62 

CS002 

1.95 

1.19 

72% 

3.30 

0.73 

3.11 

0.96 

CS003 

2.00 

1.33 

76% 

3.00 

0.82 

3.00 

0.97 

CS004 

1.73 

0.98 

90% 

3.30 

0.82 

3.47 

0.61 

CS005 

1.57 

0.85 

50% 

2.93 

0.70 

2.76 

1.03 

CS006 

1.67 

0.90 

71% 

2.63 

0.72 

3.06 

0.93 

CS007 

1.71 

0.92 

82% 

3.11 

0.81 

3.06 

0.83 

CS008 

1.73 

0.98 

77% 

3.17 

0.87 

3.11 

0.74 

CS009 

1.42 

0.69 

56% 

2.89 

0.74 

2.81 

0.66 

CS010 

1.29 

0.47 

71% 

2.63 

0.81 

2.94 

0.77 

CS011 

1.42 

0.51 

75% 

2.71 

0.91 

2.82 

0.81 

CS012 

1.59 

1.00 

69% 

2.67 

0.91 

2.83 

0.92 

CS013 

1.92 

1.02 

86% 

3.21 

0.66 

3.17 

0.71 

CS014 

1.56 

0.78 

76% 

3.39 

0.61 

3.29 

0.59 

CS015 

1.91 

1.06 

81% 

3.52 

0.59 

3.39 

0.78 

CS016 

1.55 

0.89 

84% 

3.48 

0.60 

3.18 

0.64 

CS017 

1.79 

1.02 

68% 

3.46 

0.58 

3.37 

0.68 

CS018 

1.47 

0.84 

78% 

3.45 

0.60 

3.17 

0.71 

CS019 

1.56 

0.53 

71% 

3.20 

0.79 

3.06 

0.83 

CS020 

1.33 

0.49 

75% 

3.29 

0.73 

3.12 

0.78 

Total 

1.70 

0.96 


3.19 

0.78 

3.11 

0.79 
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Key Cognitive Strategies 



Question 1 

Mean 

Score St. Dev 

Question 2 
% with 
Adequate 
Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

KCS01 

2.11 

0.96 

60% 

2.76 

0.66 

3.20 

0.77 

KCS02 

2.06 

0.68 

60% 

2.88 

0.62 

3.13 

0.64 

KCS03 

2.07 

0.70 

64% 

2.93 

0.70 

3.20 

0.68 

KCS04 

1.57 

0.51 

77% 

2.64 

0.63 

2.87 

1.06 

KCS05 

1.63 

0.62 

64% 

2.80 

0.77 

2.88 

0.96 

KCS06 

1.63 

0.52 

57% 

2.14 

0.69 

2.36 

0.93 

KCS07 

1.82 

0.87 

56% 

2.60 

0.70 

2.80 

1.15 

KCS08 

1.86 

0.66 

67% 

2.77 

0.73 

3.27 

0.70 

KCS09 

1.60 

0.51 

62% 

2.64 

1.01 

3.13 

0.74 

KCS10 

1.73 

0.59 

54% 

2.83 

0.58 

3.31 

0.63 

KCS11 

1.89 

0.81 

67% 

3.11 

0.66 

3.57 

0.51 

KCS12 

1.90 

0.91 

68% 

3.11 

0.81 

3.40 

0.63 

KCS13 

2.00 

0.82 

57% 

3.41 

0.59 

3.63 

0.50 

KCS14 

1.76 

0.75 

69% 

3.18 

0.53 

3.54 

0.52 

KCS15 

1.89 

0.78 

86% 

3.00 

0.53 

3.14 

0.86 

KCS16 

1.72 

0.75 

69% 

2.88 

0.72 

3.13 

0.74 

KCS17 

1.71 

0.69 

69% 

3.29 

0.77 

3.36 

0.74 

KCS18 

2.04 

1.02 

77% 

3.48 

0.68 

3.28 

0.75 

KCS19 

1.80 

0.63 

67% 

2.78 

0.67 

2.93 

0.83 

KCS20 

1.91 

0.70 

40% 

2.80 

0.92 

2.93 

1.00 

KCS21 

2.00 

0.63 

50% 

2.60 

1.14 

2.57 

0.94 

KCS22 

1.62 

0.65 

64% 

2.77 

0.83 

2.93 

1.07 

KCS23 

2.08 

0.90 

82% 

3.27 

0.65 

2.80 

1.01 

KCS24 

2.00 

0.82 

73% 

3.13 

0.64 

3.36 

0.74 

KCS25 

1.67 

0.50 

75% 

2.78 

0.83 

2.93 

0.92 

KCS26 

2.17 

0.75 

60% 

3.00 

0.89 

2.71 

0.91 

KCS27 

1.88 

0.99 

57% 

2.86 

0.69 

2.86 

0.86 

KCS28 

2.00 

0.53 

57% 

3.00 

0.00 

2.93 

1.00 

KCS29 

1.94 

0.97 

73% 

3.25 

0.93 

3.31 

0.79 

KCS30 

2.00 

0.58 

67% 

2.83 

0.75 

2.86 

0.95 

KCS31 

1.94 

1.00 

82% 

3.44 

0.62 

3.25 

0.68 

Total 

1.87 

0.77 


2.98 

0.75 

3.09 

0.85 
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Teaching Practices 



Question 1 Question 2 

% with 

Mean Adequate 

Score St. Dev Knowledge 

Question 3 

Mean St. 

Score Dev 

Question 4 

Mean St. 

Score Dev 

TP001 


3.59 

0.50 

3.50 

0.71 

TP002 


3.38 

0.73 

3.12 

0.99 

TP003 


3.34 

0.61 

3.38 

0.62 

TP004 


3.66 

0.48 

3.69 

0.48 

TP005 


3.55 

0.51 

3.56 

0.51 

TP006 


3.48 

0.63 

3.56 

0.63 

TP007 

N/A 

3.45 

0.69 

3.50 

0.73 

TP008 


3.48 

0.58 

3.44 

0.63 

TP009 


3.21 

0.94 

3.38 

0.62 

TP010 


3.29 

0.81 

3.38 

0.62 

TP011 


3.38 

0.78 

3.33 

0.82 

TP012 


3.24 

0.83 

3.13 

0.74 

Total 


3.40 

0.70 

3.41 

0.69 
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Appendix G: Sources 


Educational Organizations 

Computer Science Teachers Association 
Engineering Education Service Center 
Malheur ESD 

Mathematics, Engineering Science Achievement 
Northwest Regional Educational Laboratory 
Northwest Regional ESD 
OCEPT 

Oregon Council of Computer Chairs 

Oregon Council of Teachers of Mathematics 

Oregon Mathematics Education Council 

Oregon Robotics Tournament and Outreach Program 

Oregon Science Education Council 

OSU Extension 4-H BIT 

Saturday Academy 

Saturday Academy OSU 

The SMILE Program 

TWIST 

Policy Groups and Strategies, Oregon 

AEED (Academic Excellence and Economic Development Working Group), OUS 
AAWG (Access and Affordability Working Group) 

EDP (Excellence in Delivery and Productivity Working Group) 

ETIC (Oregon Engineering and Technical Industry Council) 

OMEC (Oregon Mathematics Education Council) 

ODE (Oregon Department of Education) 

OSBE (Oregon State Board of Education) 

OSBHE (Oregon State Board of Higher Education) 

Programs, Initiatives & Ciassroom Curricuia, Oregon 

Canby High School 

College Now/ Tech Prep at Central Oregon Community College 
CWW (Compressed Work Week) at PSU 
Pacts about TechPrep in Oregon 

High School - Community College Transitions in Oregon 
Hillsboro High School 

Capital Center High School Technology Institute 

STARS (Students Taking Authentic Routes to Success College and Career Exploration Program, David 
Douglas High School 
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The Center for Advanced Learning 
North Salem High School 
Oregon Small Schools Initiative 
Roseburg High School 
Sherwood High School 

SuperQuest - SAOF (Software Association of Oregon Foundation) 

OSU College of Science Outreach & Education Programs 
TeachEngineering 

Programs, Initiatives & Classroom Curricula, Other 

AP (Advanced Placement Programs) 

Amatrol: integrated technical learning systems 
Autodesk Design Academy 

CEEO (Center for Engineering Educational Outreach), Tufts University 
Infinity Project 
Intel Innovation in Education 
IB (International Baccalaureate) 

JASON Foundation for Education 
PLTW (Project Lead the Way) 

SEEK-16 (Strategies for Engineering Education K-16 Summit) 

MESA (Mathematics, Engineering, Science Achievement), Washington 

Programs & Initiatives, Co-Curricular, Oregon 

ACCESS (Alternative Career Choices for Equitable Student Success), Lane CC 

CASE (Creating Avenues, Support and Equity for Women and Minorities in Advanced Technologies), 
PCC 

Design and Discovery 

EXITE (Exploring Interests in Technology and Engineering, IBM) 

GATEway to Engineering, SWE Columbia River Chapter 

Intel NWSE (Northwest Science Expo) / ISEF (International Science and Engineering Fair) 

MESA (Mathematics, Engineering, Science Achievement), PSU, Oregon 
Oregon Building Congress Mathematics and Science Workshops 
ORTOP (Oregon Robotics Tournament Outreach Program), OUS 
Saturday Academy - PSU / OHSU 

SMILE (Science and Mathematics Investigative Learning Experiences), OSU 
STARBASE (Science and Technology Academies Reinforcing Basic Aviation and Space 
Exploration), Portland and Klamath Falls 
UO (University of Oregon) Youth Enrichment and TAG Programs 
Willamette-SAOF High School Programming Contest 
Youth Exploring Science - YES 
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Appendix H: Final Budget 


Budget 

Category 

Quarter 1 
(Apr- 
Jun 06) 

Quarter 2 
(Jul- 
Sept 06) 

Quarter 3 
(Oct - Dec 
06) 

Quarter 4 
(Jan- 
Mar 07) 

Quarter 5 
(Apr - Jun 
07) 

Quarter 6 
(Jul - Aug 
07) 

Total 

Personnel 

Expenses 








Compensation 

2,907 

17,971 

8,218 

10,356 

14,077 

4,000 

57,529 

Employee 

benefits 

1,192 

7,368 

3,369 

1,788 

3,322 

1,800 

18,839 

Document 

raters 

0 

170 

0 

0 

4,224 

0 

4,394 

Subtotal 

4,099 

25,509 

11,587 

12,144 

21,623 

5,800 

80,762 

Operating 

Expenses 








Travel 

126 

0 

0 



0 

126 

Materials, 

supplies 

424 

450 

450 

450 

20 

500 

2,294 

Telecom 

165 

165 

165 

165 

0 

0 

660 

Subtotal 

715 

615 

615 

615 

20 

500 

3,080 

Other 

Expenses 








Indirect costs 

770 

4,180 

1,952 

2,041 

3,463 

0 

12,406 

Rent/facilities 

900 

900 

900 

900 

0 

0 

3,600 

Total 

$6,484 

$31,204 

$15,055 

$15,700 

$25,106 

$6,300 

$99,849 
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